2023-02-09 14:53:31,170 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-09 14:53:31,444 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6 2023-02-09 14:53:31,461 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 14:53:31,462 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877, deleteOnExit=true 2023-02-09 14:53:31,462 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 14:53:31,462 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/test.cache.data in system properties and HBase conf 2023-02-09 14:53:31,463 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 14:53:31,463 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/hadoop.log.dir in system properties and HBase conf 2023-02-09 14:53:31,464 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 14:53:31,464 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 14:53:31,464 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 14:53:31,576 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 14:53:31,994 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 14:53:32,001 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 14:53:32,001 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 14:53:32,002 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 14:53:32,002 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 14:53:32,003 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 14:53:32,003 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 14:53:32,003 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 14:53:32,004 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 14:53:32,004 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 14:53:32,005 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/nfs.dump.dir in system properties and HBase conf 2023-02-09 14:53:32,005 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/java.io.tmpdir in system properties and HBase conf 2023-02-09 14:53:32,006 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 14:53:32,006 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 14:53:32,006 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 14:53:32,560 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 14:53:32,565 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 14:53:32,887 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 14:53:33,077 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 14:53:33,099 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:53:33,142 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:53:33,176 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/java.io.tmpdir/Jetty_localhost_44611_hdfs____.gao3k0/webapp 2023-02-09 14:53:33,327 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44611 2023-02-09 14:53:33,342 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 14:53:33,343 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 14:53:33,878 WARN [Listener at localhost/44771] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:53:33,952 WARN [Listener at localhost/44771] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 14:53:33,972 WARN [Listener at localhost/44771] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:53:33,979 INFO [Listener at localhost/44771] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:53:33,983 INFO [Listener at localhost/44771] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/java.io.tmpdir/Jetty_localhost_40511_datanode____.hh3c74/webapp 2023-02-09 14:53:34,081 INFO [Listener at localhost/44771] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40511 2023-02-09 14:53:34,361 WARN [Listener at localhost/33189] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:53:34,770 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1720a9641655affc: Processing first storage report for DS-ae56ba15-d21c-41f8-a523-22b28ac31a39 from datanode e1049c1d-88a3-49fc-8224-542a4e55a7fe 2023-02-09 14:53:34,771 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1720a9641655affc: from storage DS-ae56ba15-d21c-41f8-a523-22b28ac31a39 node DatanodeRegistration(127.0.0.1:46479, datanodeUuid=e1049c1d-88a3-49fc-8224-542a4e55a7fe, infoPort=42379, infoSecurePort=0, ipcPort=33189, storageInfo=lv=-57;cid=testClusterID;nsid=2016148506;c=1675954412651), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-09 14:53:34,771 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1720a9641655affc: Processing first storage report for DS-31065d06-82c9-49a4-8ec8-8e80b4c3e12e from datanode e1049c1d-88a3-49fc-8224-542a4e55a7fe 2023-02-09 14:53:34,771 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1720a9641655affc: from storage DS-31065d06-82c9-49a4-8ec8-8e80b4c3e12e node DatanodeRegistration(127.0.0.1:46479, datanodeUuid=e1049c1d-88a3-49fc-8224-542a4e55a7fe, infoPort=42379, infoSecurePort=0, ipcPort=33189, storageInfo=lv=-57;cid=testClusterID;nsid=2016148506;c=1675954412651), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:53:34,842 DEBUG [Listener at localhost/33189] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6 2023-02-09 14:53:34,901 INFO [Listener at localhost/33189] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/zookeeper_0, clientPort=56347, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 14:53:34,914 INFO [Listener at localhost/33189] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56347 2023-02-09 14:53:34,925 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:34,927 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:35,573 INFO [Listener at localhost/33189] util.FSUtils(479): Created version file at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d with version=8 2023-02-09 14:53:35,573 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/hbase-staging 2023-02-09 14:53:35,884 INFO [Listener at localhost/33189] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 14:53:36,351 INFO [Listener at localhost/33189] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-09 14:53:36,383 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,384 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,384 INFO [Listener at localhost/33189] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:53:36,384 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,384 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:53:36,525 INFO [Listener at localhost/33189] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:53:36,595 DEBUG [Listener at localhost/33189] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 14:53:36,685 INFO [Listener at localhost/33189] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38213 2023-02-09 14:53:36,695 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:36,697 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:36,717 INFO [Listener at localhost/33189] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38213 connecting to ZooKeeper ensemble=127.0.0.1:56347 2023-02-09 14:53:36,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:382130x0, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:53:36,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38213-0x1003992ceed0000 connected 2023-02-09 14:53:36,784 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:53:36,784 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:53:36,788 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:53:36,795 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38213 2023-02-09 14:53:36,795 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38213 2023-02-09 14:53:36,796 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38213 2023-02-09 14:53:36,796 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38213 2023-02-09 14:53:36,796 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38213 2023-02-09 14:53:36,802 INFO [Listener at localhost/33189] master.HMaster(439): hbase.rootdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d, hbase.cluster.distributed=false 2023-02-09 14:53:36,873 INFO [Listener at localhost/33189] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-09 14:53:36,873 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,873 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,874 INFO [Listener at localhost/33189] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:53:36,874 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:53:36,874 INFO [Listener at localhost/33189] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:53:36,878 INFO [Listener at localhost/33189] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:53:36,882 INFO [Listener at localhost/33189] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43455 2023-02-09 14:53:36,884 INFO [Listener at localhost/33189] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 14:53:36,891 DEBUG [Listener at localhost/33189] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 14:53:36,892 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:36,895 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:36,897 INFO [Listener at localhost/33189] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43455 connecting to ZooKeeper ensemble=127.0.0.1:56347 2023-02-09 14:53:36,901 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:434550x0, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:53:36,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43455-0x1003992ceed0001 connected 2023-02-09 14:53:36,902 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:53:36,903 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:53:36,904 DEBUG [Listener at localhost/33189] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:53:36,905 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43455 2023-02-09 14:53:36,905 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43455 2023-02-09 14:53:36,905 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43455 2023-02-09 14:53:36,906 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43455 2023-02-09 14:53:36,906 DEBUG [Listener at localhost/33189] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43455 2023-02-09 14:53:36,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:36,916 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 14:53:36,917 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:36,937 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:53:36,937 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:53:36,937 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:36,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 14:53:36,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38213,1675954415718 from backup master directory 2023-02-09 14:53:36,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 14:53:36,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:36,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 14:53:36,943 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:53:36,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:36,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 14:53:36,947 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 14:53:37,027 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/hbase.id with ID: 0a453c20-391c-4d99-944f-dbd11d146d32 2023-02-09 14:53:37,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:37,083 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:37,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x270393bd to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:37,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1002383b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:37,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 14:53:37,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 14:53:37,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 14:53:37,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 14:53:37,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:53:37,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:53:37,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:53:37,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store-tmp 2023-02-09 14:53:37,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:37,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 14:53:37,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:53:37,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:53:37,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 14:53:37,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:53:37,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:53:37,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675954417252Disabling compacts and flushes for region at 1675954417252Disabling writes for close at 1675954417253 (+1 ms)Writing region close event to WAL at 1675954417253Closed at 1675954417253 2023-02-09 14:53:37,254 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/WALs/jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:37,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38213%2C1675954415718, suffix=, logDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/WALs/jenkins-hbase4.apache.org,38213,1675954415718, archiveDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/oldWALs, maxLogs=10 2023-02-09 14:53:37,325 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK] 2023-02-09 14:53:37,333 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:53:37,389 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/WALs/jenkins-hbase4.apache.org,38213,1675954415718/jenkins-hbase4.apache.org%2C38213%2C1675954415718.1675954417284 2023-02-09 14:53:37,390 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK]] 2023-02-09 14:53:37,391 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:37,391 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:37,394 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,450 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,457 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 14:53:37,479 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 14:53:37,491 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:37,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:53:37,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:53:37,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75162481, jitterRate=0.12000824511051178}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:53:37,518 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675954417420Initializing all the Stores at 1675954417423 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954417423Cleaning up temporary data from old regions at 1675954417504 (+81 ms)Cleaning up detritus from prior splits at 1675954417505 (+1 ms)Region opened successfully at 1675954417518 (+13 ms) 2023-02-09 14:53:37,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 14:53:37,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-09 14:53:37,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-09 14:53:37,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:53:37,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 14:53:37,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-09 14:53:37,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 14:53:37,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 14:53:37,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 14:53:37,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 14:53:37,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 14:53:37,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 14:53:37,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 14:53:37,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 14:53:37,644 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:37,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 14:53:37,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 14:53:37,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 14:53:37,661 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:53:37,661 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:53:37,662 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:37,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,38213,1675954415718, sessionid=0x1003992ceed0000, setting cluster-up flag (Was=false) 2023-02-09 14:53:37,676 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:37,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 14:53:37,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:37,689 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:37,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 14:53:37,696 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:37,698 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp 2023-02-09 14:53:37,710 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(952): ClusterId : 0a453c20-391c-4d99-944f-dbd11d146d32 2023-02-09 14:53:37,714 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 14:53:37,719 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 14:53:37,719 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 14:53:37,721 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 14:53:37,722 DEBUG [RS:0;jenkins-hbase4:43455] zookeeper.ReadOnlyZKClient(139): Connect 0x13a2099f to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:37,727 DEBUG [RS:0;jenkins-hbase4:43455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d65f59b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:37,728 DEBUG [RS:0;jenkins-hbase4:43455] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@195a62ef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:53:37,752 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43455 2023-02-09 14:53:37,756 INFO [RS:0;jenkins-hbase4:43455] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 14:53:37,756 INFO [RS:0;jenkins-hbase4:43455] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 14:53:37,756 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 14:53:37,759 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38213,1675954415718 with isa=jenkins-hbase4.apache.org/172.31.14.131:43455, startcode=1675954416872 2023-02-09 14:53:37,774 DEBUG [RS:0;jenkins-hbase4:43455] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 14:53:37,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 14:53:37,822 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60095, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 14:53:37,825 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:37,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:53:37,827 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:37,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954447828 2023-02-09 14:53:37,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-09 14:53:37,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 14:53:37,835 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 14:53:37,835 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 14:53:37,841 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 14:53:37,843 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 14:53:37,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 14:53:37,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 14:53:37,853 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 14:53:37,853 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 14:53:37,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:37,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 14:53:37,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 14:53:37,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 14:53:37,860 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(2829): Master is not running yet 2023-02-09 14:53:37,861 WARN [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-09 14:53:37,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 14:53:37,862 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 14:53:37,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954417864,5,FailOnTimeoutGroup] 2023-02-09 14:53:37,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954417864,5,FailOnTimeoutGroup] 2023-02-09 14:53:37,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:37,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 14:53:37,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:37,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:37,873 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:37,874 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:37,874 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d 2023-02-09 14:53:37,962 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38213,1675954415718 with isa=jenkins-hbase4.apache.org/172.31.14.131:43455, startcode=1675954416872 2023-02-09 14:53:37,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:37,973 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d 2023-02-09 14:53:37,974 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44771 2023-02-09 14:53:37,974 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 14:53:37,979 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:53:37,979 DEBUG [RS:0;jenkins-hbase4:43455] zookeeper.ZKUtil(162): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:37,980 WARN [RS:0;jenkins-hbase4:43455] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:53:37,980 INFO [RS:0;jenkins-hbase4:43455] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:53:37,981 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/WALs/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:37,981 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43455,1675954416872] 2023-02-09 14:53:37,989 DEBUG [RS:0;jenkins-hbase4:43455] zookeeper.ZKUtil(162): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:37,999 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 14:53:38,007 INFO [RS:0;jenkins-hbase4:43455] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 14:53:38,026 INFO [RS:0;jenkins-hbase4:43455] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 14:53:38,030 INFO [RS:0;jenkins-hbase4:43455] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:53:38,030 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,031 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 14:53:38,037 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,037 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,038 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,039 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,039 DEBUG [RS:0;jenkins-hbase4:43455] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:53:38,040 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,040 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,040 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,055 INFO [RS:0;jenkins-hbase4:43455] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 14:53:38,057 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43455,1675954416872-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,073 INFO [RS:0;jenkins-hbase4:43455] regionserver.Replication(203): jenkins-hbase4.apache.org,43455,1675954416872 started 2023-02-09 14:53:38,073 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43455,1675954416872, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43455, sessionid=0x1003992ceed0001 2023-02-09 14:53:38,073 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 14:53:38,073 DEBUG [RS:0;jenkins-hbase4:43455] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:38,073 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43455,1675954416872' 2023-02-09 14:53:38,073 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:38,074 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:38,074 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 14:53:38,074 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 14:53:38,075 DEBUG [RS:0;jenkins-hbase4:43455] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:38,075 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43455,1675954416872' 2023-02-09 14:53:38,075 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:53:38,075 DEBUG [RS:0;jenkins-hbase4:43455] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:53:38,075 DEBUG [RS:0;jenkins-hbase4:43455] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 14:53:38,076 INFO [RS:0;jenkins-hbase4:43455] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 14:53:38,076 INFO [RS:0;jenkins-hbase4:43455] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 14:53:38,186 INFO [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43455%2C1675954416872, suffix=, logDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/WALs/jenkins-hbase4.apache.org,43455,1675954416872, archiveDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/oldWALs, maxLogs=32 2023-02-09 14:53:38,205 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK] 2023-02-09 14:53:38,208 INFO [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/WALs/jenkins-hbase4.apache.org,43455,1675954416872/jenkins-hbase4.apache.org%2C43455%2C1675954416872.1675954418189 2023-02-09 14:53:38,209 DEBUG [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK]] 2023-02-09 14:53:38,286 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:38,288 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 14:53:38,291 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/info 2023-02-09 14:53:38,291 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 14:53:38,292 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,293 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 14:53:38,295 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:53:38,296 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 14:53:38,296 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,297 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 14:53:38,299 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/table 2023-02-09 14:53:38,299 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 14:53:38,300 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,301 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740 2023-02-09 14:53:38,302 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740 2023-02-09 14:53:38,306 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 14:53:38,308 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 14:53:38,311 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:53:38,312 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72111963, jitterRate=0.07455198466777802}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 14:53:38,312 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675954418286Initializing all the Stores at 1675954418287 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954418288 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954418288Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954418288Cleaning up temporary data from old regions at 1675954418304 (+16 ms)Cleaning up detritus from prior splits at 1675954418305 (+1 ms)Region opened successfully at 1675954418312 (+7 ms) 2023-02-09 14:53:38,312 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 14:53:38,312 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 14:53:38,312 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 14:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 14:53:38,313 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 14:53:38,313 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 14:53:38,314 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675954418312Disabling compacts and flushes for region at 1675954418312Disabling writes for close at 1675954418313 (+1 ms)Writing region close event to WAL at 1675954418313Closed at 1675954418313 2023-02-09 14:53:38,319 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 14:53:38,319 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 14:53:38,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 14:53:38,340 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 14:53:38,342 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 14:53:38,494 DEBUG [jenkins-hbase4:38213] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-09 14:53:38,497 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43455,1675954416872, state=OPENING 2023-02-09 14:53:38,504 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 14:53:38,505 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:38,506 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 14:53:38,509 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:38,695 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:38,697 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:53:38,701 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50498, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:53:38,712 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 14:53:38,712 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:53:38,715 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43455%2C1675954416872.meta, suffix=.meta, logDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/WALs/jenkins-hbase4.apache.org,43455,1675954416872, archiveDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/oldWALs, maxLogs=32 2023-02-09 14:53:38,731 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK] 2023-02-09 14:53:38,737 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/WALs/jenkins-hbase4.apache.org,43455,1675954416872/jenkins-hbase4.apache.org%2C43455%2C1675954416872.meta.1675954418716.meta 2023-02-09 14:53:38,737 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46479,DS-ae56ba15-d21c-41f8-a523-22b28ac31a39,DISK]] 2023-02-09 14:53:38,737 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:38,740 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 14:53:38,757 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 14:53:38,762 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 14:53:38,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 14:53:38,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:38,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 14:53:38,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 14:53:38,771 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 14:53:38,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/info 2023-02-09 14:53:38,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/info 2023-02-09 14:53:38,774 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 14:53:38,774 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 14:53:38,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:53:38,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:53:38,777 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 14:53:38,778 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,778 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 14:53:38,779 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/table 2023-02-09 14:53:38,780 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/table 2023-02-09 14:53:38,780 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 14:53:38,781 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:38,782 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740 2023-02-09 14:53:38,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740 2023-02-09 14:53:38,788 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 14:53:38,790 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 14:53:38,791 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75344579, jitterRate=0.12272171676158905}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 14:53:38,791 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675954418768Writing region info on filesystem at 1675954418769 (+1 ms)Initializing all the Stores at 1675954418770 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954418771 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954418771Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954418771Cleaning up temporary data from old regions at 1675954418786 (+15 ms)Cleaning up detritus from prior splits at 1675954418787 (+1 ms)Running coprocessor post-open hooks at 1675954418791 (+4 ms)Region opened successfully at 1675954418791 2023-02-09 14:53:38,800 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675954418687 2023-02-09 14:53:38,817 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 14:53:38,817 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 14:53:38,818 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43455,1675954416872, state=OPEN 2023-02-09 14:53:38,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 14:53:38,822 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 14:53:38,826 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 14:53:38,826 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43455,1675954416872 in 313 msec 2023-02-09 14:53:38,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 14:53:38,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 500 msec 2023-02-09 14:53:38,836 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.0890 sec 2023-02-09 14:53:38,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675954416933, completionTime=-1 2023-02-09 14:53:38,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-09 14:53:38,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:53:38,896 DEBUG [hconnection-0x23c84b95-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:53:38,900 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50514, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:53:38,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-09 14:53:38,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954478917 2023-02-09 14:53:38,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954538917 2023-02-09 14:53:38,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 80 msec 2023-02-09 14:53:38,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38213,1675954415718-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38213,1675954415718-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38213,1675954415718-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38213, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 14:53:38,946 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 14:53:38,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 14:53:38,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 14:53:38,964 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 14:53:38,966 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:53:38,968 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:53:38,988 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:38,991 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7 empty. 2023-02-09 14:53:38,991 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:38,991 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 14:53:39,411 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:39,414 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => e5181eaff4f036103b5f31d3d6a127e7, NAME => 'hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:53:39,826 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:39,826 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing e5181eaff4f036103b5f31d3d6a127e7, disabling compactions & flushes 2023-02-09 14:53:39,826 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:39,826 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:39,826 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. after waiting 0 ms 2023-02-09 14:53:39,827 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:39,827 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:39,827 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for e5181eaff4f036103b5f31d3d6a127e7: Waiting for close lock at 1675954419826Disabling compacts and flushes for region at 1675954419826Disabling writes for close at 1675954419827 (+1 ms)Writing region close event to WAL at 1675954419827Closed at 1675954419827 2023-02-09 14:53:39,831 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:53:39,846 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954419834"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954419834"}]},"ts":"1675954419834"} 2023-02-09 14:53:39,870 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:53:39,871 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:53:39,875 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954419872"}]},"ts":"1675954419872"} 2023-02-09 14:53:39,879 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 14:53:39,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e5181eaff4f036103b5f31d3d6a127e7, ASSIGN}] 2023-02-09 14:53:39,891 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e5181eaff4f036103b5f31d3d6a127e7, ASSIGN 2023-02-09 14:53:39,893 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e5181eaff4f036103b5f31d3d6a127e7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:53:40,044 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e5181eaff4f036103b5f31d3d6a127e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:40,044 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954420043"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954420043"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954420043"}]},"ts":"1675954420043"} 2023-02-09 14:53:40,048 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e5181eaff4f036103b5f31d3d6a127e7, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:40,208 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:40,208 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e5181eaff4f036103b5f31d3d6a127e7, NAME => 'hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:40,209 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,209 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:40,209 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,209 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,211 INFO [StoreOpener-e5181eaff4f036103b5f31d3d6a127e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,213 DEBUG [StoreOpener-e5181eaff4f036103b5f31d3d6a127e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/info 2023-02-09 14:53:40,213 DEBUG [StoreOpener-e5181eaff4f036103b5f31d3d6a127e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/info 2023-02-09 14:53:40,214 INFO [StoreOpener-e5181eaff4f036103b5f31d3d6a127e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5181eaff4f036103b5f31d3d6a127e7 columnFamilyName info 2023-02-09 14:53:40,215 INFO [StoreOpener-e5181eaff4f036103b5f31d3d6a127e7-1] regionserver.HStore(310): Store=e5181eaff4f036103b5f31d3d6a127e7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:40,216 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,217 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,221 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:53:40,224 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:53:40,225 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e5181eaff4f036103b5f31d3d6a127e7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64614164, jitterRate=-0.03717392683029175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:53:40,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e5181eaff4f036103b5f31d3d6a127e7: Running coprocessor pre-open hook at 1675954420209Writing region info on filesystem at 1675954420209Initializing all the Stores at 1675954420211 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954420211Cleaning up temporary data from old regions at 1675954420218 (+7 ms)Cleaning up detritus from prior splits at 1675954420218Running coprocessor post-open hooks at 1675954420225 (+7 ms)Region opened successfully at 1675954420225 2023-02-09 14:53:40,227 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7., pid=6, masterSystemTime=1675954420202 2023-02-09 14:53:40,231 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:40,231 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:53:40,232 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e5181eaff4f036103b5f31d3d6a127e7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:40,233 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954420231"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954420231"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954420231"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954420231"}]},"ts":"1675954420231"} 2023-02-09 14:53:40,239 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 14:53:40,239 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e5181eaff4f036103b5f31d3d6a127e7, server=jenkins-hbase4.apache.org,43455,1675954416872 in 188 msec 2023-02-09 14:53:40,243 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 14:53:40,243 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e5181eaff4f036103b5f31d3d6a127e7, ASSIGN in 350 msec 2023-02-09 14:53:40,244 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:53:40,245 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954420244"}]},"ts":"1675954420244"} 2023-02-09 14:53:40,247 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 14:53:40,251 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:53:40,253 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.2960 sec 2023-02-09 14:53:40,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 14:53:40,268 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:53:40,268 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:53:40,301 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 14:53:40,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:53:40,323 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-02-09 14:53:40,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:53:40,344 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:53:40,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-02-09 14:53:40,359 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 14:53:40,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 14:53:40,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.420sec 2023-02-09 14:53:40,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 14:53:40,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 14:53:40,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 14:53:40,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38213,1675954415718-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 14:53:40,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38213,1675954415718-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 14:53:40,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 14:53:40,416 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(139): Connect 0x57919e99 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:40,421 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73d84839, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:40,434 DEBUG [hconnection-0x346dc13e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:53:40,446 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50526, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:53:40,456 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:53:40,457 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(139): Connect 0x480b078d to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:40,461 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15776577, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:40,461 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:53:40,482 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=349, ProcessCount=168, AvailableMemoryMB=4096 2023-02-09 14:53:40,485 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:40,486 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:40,493 DEBUG [Listener at localhost/33189] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:53:40,496 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58548, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:53:40,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:53:40,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:40,511 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:53:40,513 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:53:40,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-09 14:53:40,517 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,518 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed empty. 2023-02-09 14:53:40,520 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,520 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-09 14:53:40,523 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:40,523 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:40,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 14:53:40,536 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:40,537 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 482df8fbf041dd5fc164a0c315c9e3ed, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:53:40,549 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:40,549 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 482df8fbf041dd5fc164a0c315c9e3ed, disabling compactions & flushes 2023-02-09 14:53:40,549 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,549 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,549 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. after waiting 0 ms 2023-02-09 14:53:40,549 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,549 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,550 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 482df8fbf041dd5fc164a0c315c9e3ed: Waiting for close lock at 1675954420549Disabling compacts and flushes for region at 1675954420549Disabling writes for close at 1675954420549Writing region close event to WAL at 1675954420549Closed at 1675954420549 2023-02-09 14:53:40,554 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:53:40,555 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954420555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954420555"}]},"ts":"1675954420555"} 2023-02-09 14:53:40,558 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:53:40,559 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:53:40,559 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954420559"}]},"ts":"1675954420559"} 2023-02-09 14:53:40,562 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-09 14:53:40,565 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, ASSIGN}] 2023-02-09 14:53:40,567 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, ASSIGN 2023-02-09 14:53:40,568 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:53:40,629 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:40,629 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:40,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 14:53:40,719 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:40,720 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954420719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954420719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954420719"}]},"ts":"1675954420719"} 2023-02-09 14:53:40,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:40,831 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:40,831 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:40,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 14:53:40,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 482df8fbf041dd5fc164a0c315c9e3ed, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:40,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:40,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,885 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,887 DEBUG [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:40,887 DEBUG [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:40,890 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 482df8fbf041dd5fc164a0c315c9e3ed columnFamilyName family1 2023-02-09 14:53:40,893 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] regionserver.HStore(310): Store=482df8fbf041dd5fc164a0c315c9e3ed/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:40,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:40,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:53:40,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 482df8fbf041dd5fc164a0c315c9e3ed; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62700210, jitterRate=-0.06569406390190125}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:53:40,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 482df8fbf041dd5fc164a0c315c9e3ed: Running coprocessor pre-open hook at 1675954420882Writing region info on filesystem at 1675954420882Initializing all the Stores at 1675954420883 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954420884 (+1 ms)Cleaning up temporary data from old regions at 1675954420896 (+12 ms)Cleaning up detritus from prior splits at 1675954420897 (+1 ms)Running coprocessor post-open hooks at 1675954420905 (+8 ms)Region opened successfully at 1675954420905 2023-02-09 14:53:40,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., pid=11, masterSystemTime=1675954420876 2023-02-09 14:53:40,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:40,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:40,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954420911"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954420911"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954420911"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954420911"}]},"ts":"1675954420911"} 2023-02-09 14:53:40,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 14:53:40,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872 in 192 msec 2023-02-09 14:53:40,921 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 14:53:40,922 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, ASSIGN in 353 msec 2023-02-09 14:53:40,923 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:53:40,923 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954420923"}]},"ts":"1675954420923"} 2023-02-09 14:53:40,926 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-09 14:53:40,930 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:53:40,932 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 422 msec 2023-02-09 14:53:41,133 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:41,134 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:41,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 14:53:41,135 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-09 14:53:41,152 DEBUG [hconnection-0x7a3c26d9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:53:41,155 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50530, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:53:41,294 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:41,294 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:41,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:41,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,304 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:41,304 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:41,305 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:41,305 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:41,305 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,306 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,307 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:41,307 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:41,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:41,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:41,308 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:41,308 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:41,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:41,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:41,311 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:41,311 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:41,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:41,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:41,313 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:41,313 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:41,314 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:41,314 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:53:41,850 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130460609ba88d447c6a39b88b199edf078 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130460609ba88d447c6a39b88b199edf078 2023-02-09 14:53:42,266 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/2cc17f8f84174aaba112e1ae26bd6191 2023-02-09 14:53:42,278 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/2cc17f8f84174aaba112e1ae26bd6191 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191 2023-02-09 14:53:42,287 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-09 14:53:42,290 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 975ms, sequenceid=5, compaction requested=false 2023-02-09 14:53:42,291 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:42,294 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:42,294 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:42,295 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:42,295 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,295 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:42,295 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:42,304 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,304 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:42,305 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:42,305 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:42,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:42,307 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:42,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72eb9f56[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:42,307 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,389 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:42,389 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,390 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:42,390 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:42,390 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:42,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:42,475 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:42,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:42,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:42,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:42,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:42,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:42,481 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:42,481 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:42,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:42,482 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:42,482 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,482 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:42,567 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,567 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:42,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:42,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,568 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:42,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:42,569 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:42,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:42,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:42,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,572 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:42,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:42,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:42,620 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,620 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:42,621 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:42,621 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,622 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:42,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:42,623 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:42,623 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:42,724 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:42,724 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:42,724 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:42,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:42,740 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:42,741 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:42,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:42,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:42,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,745 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:42,745 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:42,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:42,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:42,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,820 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:42,820 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:42,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:42,821 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,821 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:42,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:42,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:42,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:42,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:42,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:42,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:42,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:42,823 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:42,823 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:42,824 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:53:43,257 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285a128ad9fc5440e38cba17b0be21a415 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285a128ad9fc5440e38cba17b0be21a415 2023-02-09 14:53:43,670 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/49c4d4ee2f99488481023cd689def5a5 2023-02-09 14:53:43,680 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/49c4d4ee2f99488481023cd689def5a5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5 2023-02-09 14:53:43,688 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:53:43,690 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 866ms, sequenceid=9, compaction requested=false 2023-02-09 14:53:43,691 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:43,691 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:43,691 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:43,692 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:43,692 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,692 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:43,692 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:43,694 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:43,694 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,696 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,696 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:43,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:43,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:43,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ee5a2b1[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:43,697 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:43,697 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,699 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,699 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:43,699 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:43,699 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:43,699 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,699 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:43,701 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,701 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:43,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:43,701 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:43,701 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:43,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:43,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:43,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:43,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:43,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:43,706 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:43,706 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:43,706 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:43,706 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,706 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:43,713 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,713 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,713 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:43,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:43,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:43,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:43,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:43,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:43,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,720 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:43,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,724 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:43,725 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:43,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:43,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:43,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:43,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:43,725 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,726 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,726 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,726 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:43,726 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:43,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:43,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:43,826 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:43,826 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:43,826 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:43,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:43,836 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:43,836 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:43,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:43,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:43,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,841 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:43,841 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:43,841 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:43,841 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:43,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,843 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:43,843 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:43,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:43,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:43,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:43,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,845 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:43,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,845 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:43,845 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:43,846 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:43,846 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:43,846 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:43,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:43,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:43,847 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:43,847 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:43,847 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:44,283 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e21e16fa3ade444caa960123853c2ce7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21e16fa3ade444caa960123853c2ce7 2023-02-09 14:53:44,288 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:53:44,315 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/ace9169066374876af3f1ddc9d7f0364 2023-02-09 14:53:44,336 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/ace9169066374876af3f1ddc9d7f0364 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364 2023-02-09 14:53:44,348 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 14:53:44,349 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 502ms, sequenceid=13, compaction requested=true 2023-02-09 14:53:44,349 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:44,349 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,350 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:44,350 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:44,350 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,350 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:44,350 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:44,353 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,354 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,354 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:44,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:44,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52425c4d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:44,357 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:44,357 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,359 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:44,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:44,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:44,368 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,368 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:44,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,371 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,371 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:44,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:44,381 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:44,381 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:44,381 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:44,381 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,381 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:44,384 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,384 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,384 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,385 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,390 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,390 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 14:53:44,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,392 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,396 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:44,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,397 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,397 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,397 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:44,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,398 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:44,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,498 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:44,498 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,498 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:44,509 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,509 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:44,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:44,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,517 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,517 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:44,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:44,519 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,519 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:44,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,525 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:44,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:44,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:44,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:44,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:44,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:44,527 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:44,563 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057c9ba77c1d504a0d8bcb850facfc2da6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057c9ba77c1d504a0d8bcb850facfc2da6 2023-02-09 14:53:44,580 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/e99bbaa811034bd9b13db57b307660ff 2023-02-09 14:53:44,589 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/e99bbaa811034bd9b13db57b307660ff as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff 2023-02-09 14:53:44,601 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 14:53:44,602 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 75ms, sequenceid=17, compaction requested=true 2023-02-09 14:53:44,602 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:44,602 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:44,607 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,608 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,609 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,609 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:44,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:44,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6421f178[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:44,611 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:44,611 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,612 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,613 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:44,613 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:44,613 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:44,613 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,613 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:44,615 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,615 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:44,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,615 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,615 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:44,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:44,620 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:44,620 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:44,620 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:44,620 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,620 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:44,623 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,623 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,623 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,625 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,627 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,633 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,633 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,633 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,633 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,633 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,634 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,634 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,634 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,634 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,634 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:44,636 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:44,636 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:44,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:44,737 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:44,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:44,745 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,746 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:44,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:44,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,749 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,749 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,750 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:44,750 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:44,751 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,751 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,752 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,752 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,752 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,753 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:44,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:44,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:44,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,759 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:44,759 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,760 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:44,760 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:44,760 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:44,760 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,760 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:44,761 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:44,789 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122803d39e80496043b3a8255a0a845c21ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122803d39e80496043b3a8255a0a845c21ed 2023-02-09 14:53:44,802 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/5fafbd5707184371ba6d07cb79be75ff 2023-02-09 14:53:44,820 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/5fafbd5707184371ba6d07cb79be75ff as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff 2023-02-09 14:53:44,831 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 14:53:44,832 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 71ms, sequenceid=21, compaction requested=true 2023-02-09 14:53:44,833 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:44,833 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:44,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:44,840 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,841 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,841 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:44,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:44,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a1ec385[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:44,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:44,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,845 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:44,845 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,846 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:44,847 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:44,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:44,851 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:44,852 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:44,852 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:44,852 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,852 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:44,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:44,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:44,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,855 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,857 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,861 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:44,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:44,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:44,863 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:44,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:44,963 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:44,963 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,964 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:44,974 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:44,975 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:44,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:44,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:44,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,979 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:44,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:44,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:44,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,981 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,981 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:44,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:44,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:44,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:44,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,983 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,983 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:44,983 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:44,983 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:44,983 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:44,984 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:44,984 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:44,984 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:44,985 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:45,412 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512318f19ec1e2a5a4b538c8fbd82109c2ef1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318f19ec1e2a5a4b538c8fbd82109c2ef1 2023-02-09 14:53:45,451 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/728ef16b6dbe45519f1721adbe4a6e1c 2023-02-09 14:53:45,459 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/728ef16b6dbe45519f1721adbe4a6e1c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c 2023-02-09 14:53:45,468 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 14:53:45,469 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 484ms, sequenceid=25, compaction requested=true 2023-02-09 14:53:45,469 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:45,469 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:45,469 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:45,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:45,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:45,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:45,472 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,472 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:45,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:45,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59fd105d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:45,474 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:45,474 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,476 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,476 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:45,476 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:45,476 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:45,476 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,476 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:45,484 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,484 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:45,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:45,484 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:45,484 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:45,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:45,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:45,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:45,488 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:45,488 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:45,488 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:45,488 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,488 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:45,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:45,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:45,491 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,492 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:45,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,495 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,495 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,495 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,495 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,496 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:45,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:45,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:45,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:45,496 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,496 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:45,496 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,496 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:45,496 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:45,497 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:45,497 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:45,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:45,597 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:45,597 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:45,597 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:45,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:45,605 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:45,605 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:45,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:45,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:45,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,609 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:45,609 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:45,609 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:45,609 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:45,610 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,610 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,611 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,611 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:45,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:45,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:45,612 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:45,612 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:45,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:45,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,613 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,613 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:45,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:45,614 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:45,614 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:45,614 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:45,614 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:45,614 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:45,614 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:45,614 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:45,615 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:45,878 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 14:53:45,878 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 14:53:46,037 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101b4b759d182fa46c98c4d4a19ed6851ad to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b4b759d182fa46c98c4d4a19ed6851ad 2023-02-09 14:53:46,472 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/635fe7e86951412089176d4f847c0441 2023-02-09 14:53:46,480 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/635fe7e86951412089176d4f847c0441 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441 2023-02-09 14:53:46,489 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 14:53:46,491 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 876ms, sequenceid=29, compaction requested=true 2023-02-09 14:53:46,491 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:46,491 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:46,495 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,495 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:46,495 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,495 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191d8f1b[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:46,497 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:46,497 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,498 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,498 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:46,498 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:46,498 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,498 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:46,498 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:46,500 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:46,500 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,500 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:46,500 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:46,503 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:46,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:46,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:46,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,503 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:46,505 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,505 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:46,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,505 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:46,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:46,506 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,507 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:46,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:46,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:46,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:46,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:46,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:46,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,515 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:46,515 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:46,615 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:46,615 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:46,615 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:46,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:46,622 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:46,622 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:46,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:46,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:46,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,626 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:46,626 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:46,626 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:46,626 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:46,627 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,627 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,628 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:46,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:46,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:46,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:46,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:46,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:46,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:46,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:46,630 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:46,630 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:46,630 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:46,630 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:46,630 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:46,630 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:46,630 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:46,631 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:46,650 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010343b2a515be50488c8e443739b5d41536 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010343b2a515be50488c8e443739b5d41536 2023-02-09 14:53:47,059 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/4c0b0796ac0f48d89b7b9819a05c0dd8 2023-02-09 14:53:47,067 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/4c0b0796ac0f48d89b7b9819a05c0dd8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8 2023-02-09 14:53:47,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 14:53:47,075 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 444ms, sequenceid=33, compaction requested=true 2023-02-09 14:53:47,076 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:47,076 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:47,076 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:47,076 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:47,076 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,076 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:47,076 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:47,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,078 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:47,079 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,079 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:47,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:47,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72dc1771[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:47,081 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:47,081 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,082 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:47,082 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,082 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:47,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:47,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:47,088 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:47,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:47,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:47,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,088 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:47,090 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,090 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:47,090 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:47,091 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,092 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,094 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,094 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:47,094 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:47,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:47,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:47,094 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:47,094 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,094 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:47,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:47,095 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:47,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:47,195 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:47,196 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:47,196 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:47,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:47,203 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:47,203 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:47,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:47,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:47,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,207 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:47,207 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:47,208 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:47,208 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:47,208 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,208 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,209 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:47,209 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:47,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:47,210 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:47,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,210 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:47,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:47,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:47,213 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:47,213 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:47,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:47,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:47,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:47,214 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:47,214 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:47,214 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:47,635 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907789f80c557f840e8a5a43f625068a9f6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907789f80c557f840e8a5a43f625068a9f6 2023-02-09 14:53:48,046 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/f16b37dafd184dc2b2c820e587208f6f 2023-02-09 14:53:48,054 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/f16b37dafd184dc2b2c820e587208f6f as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f 2023-02-09 14:53:48,060 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 14:53:48,061 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 847ms, sequenceid=37, compaction requested=true 2023-02-09 14:53:48,061 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:48,061 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:48,061 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:48,062 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:48,062 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,062 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:48,062 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:48,064 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,064 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:48,064 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:48,064 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:48,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:48,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:48,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:48,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e62f044[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:48,066 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:48,066 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,067 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:48,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:48,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:48,067 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,067 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:48,069 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:48,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:48,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:48,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:48,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:48,072 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:48,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:48,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:48,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,072 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:48,075 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,075 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:48,075 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:48,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:48,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:48,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:48,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,079 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:48,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:48,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:48,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:48,080 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:48,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:48,080 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,081 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,081 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:48,081 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:48,081 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:48,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:48,181 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:48,181 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:48,181 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:48,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:48,189 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:48,189 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:48,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:48,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:48,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,192 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:48,192 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:48,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 14:53:48,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 14:53:48,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,195 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:48,195 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:48,195 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:48,195 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:48,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 14:53:48,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 14:53:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 14:53:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:48,198 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:48,198 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:48,199 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. started... 2023-02-09 14:53:48,199 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 482df8fbf041dd5fc164a0c315c9e3ed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:48,619 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a8e9377a86a648738d206d845684ecd5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8e9377a86a648738d206d845684ecd5 2023-02-09 14:53:49,028 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/dd5868b996cf4d8893ee7f07a225da36 2023-02-09 14:53:49,036 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/dd5868b996cf4d8893ee7f07a225da36 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36 2023-02-09 14:53:49,042 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 14:53:49,044 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 482df8fbf041dd5fc164a0c315c9e3ed in 845ms, sequenceid=41, compaction requested=true 2023-02-09 14:53:49,044 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:49,044 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:49,044 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:49,044 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:49,045 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,045 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 14:53:49,045 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 14:53:49,046 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,046 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:49,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:49,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:49,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:49,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:49,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 14:53:49,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fac6e2d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:49,048 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 14:53:49,048 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,050 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,050 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 14:53:49,050 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 14:53:49,050 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,050 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:49,050 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 14:53:49,055 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:49,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 14:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:49,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 14:53:49,058 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 14:53:49,059 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:49,059 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:49,059 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,059 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:49,060 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,060 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,060 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:49,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:49,061 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,062 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,066 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:49,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 14:53:49,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:49,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:49,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,066 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:49,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:49,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:49,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 14:53:49,067 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:49,167 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:49,167 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:49,167 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:49,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:49,173 DEBUG [Listener at localhost/33189] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:53:49,176 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42582, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:53:49,177 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:49,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:49,193 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:49,194 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:53:49,194 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955029194 2023-02-09 14:53:49,196 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:53:49,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 14:53:49,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:49,200 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x6059c336 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:49,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19baf001, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:49,205 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,074 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103147a80dbf1d14e239cef87b811ae872d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103147a80dbf1d14e239cef87b811ae872d 2023-02-09 14:53:50,076 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:50,094 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:53:50,096 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51600, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:53:50,106 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:50,106 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:50,108 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:53:50,110 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42594, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:53:50,144 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a0a4aae50f544a39ae6b48cdb2127375 first=Optional[r3] last=Optional[r6] 2023-02-09 14:53:50,156 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a0a4aae50f544a39ae6b48cdb2127375}] 2023-02-09 14:53:50,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a0a4aae50f544a39ae6b48cdb2127375 for inclusion in 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 14:53:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:53:50,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:50,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a0a4aae50f544a39ae6b48cdb2127375 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7t4np8ctfo2hpvtctohdj9002m4deru1ao17vfrp3eakv8rola49tiuthaitiv8n/family1/a0a4aae50f544a39ae6b48cdb2127375 2023-02-09 14:53:50,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7t4np8ctfo2hpvtctohdj9002m4deru1ao17vfrp3eakv8rola49tiuthaitiv8n/family1/a0a4aae50f544a39ae6b48cdb2127375 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_ 2023-02-09 14:53:50,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7t4np8ctfo2hpvtctohdj9002m4deru1ao17vfrp3eakv8rola49tiuthaitiv8n/family1/a0a4aae50f544a39ae6b48cdb2127375 into 482df8fbf041dd5fc164a0c315c9e3ed/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_ - updating store file list. 2023-02-09 14:53:50,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_ into 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:50,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7t4np8ctfo2hpvtctohdj9002m4deru1ao17vfrp3eakv8rola49tiuthaitiv8n/family1/a0a4aae50f544a39ae6b48cdb2127375 into 482df8fbf041dd5fc164a0c315c9e3ed/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_) 2023-02-09 14:53:50,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__7t4np8ctfo2hpvtctohdj9002m4deru1ao17vfrp3eakv8rola49tiuthaitiv8n/family1/a0a4aae50f544a39ae6b48cdb2127375 2023-02-09 14:53:50,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:53:50,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:53:50,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 482df8fbf041dd5fc164a0c315c9e3ed/family1 is initiating minor compaction 2023-02-09 14:53:50,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@40e892ca; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:53:50,203 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 482df8fbf041dd5fc164a0c315c9e3ed/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:50,203 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp, totalSize=50.2 K 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 49c4d4ee2f99488481023cd689def5a5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting ace9169066374876af3f1ddc9d7f0364, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting e99bbaa811034bd9b13db57b307660ff, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 5fafbd5707184371ba6d07cb79be75ff, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 728ef16b6dbe45519f1721adbe4a6e1c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 635fe7e86951412089176d4f847c0441, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4c0b0796ac0f48d89b7b9819a05c0dd8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting f16b37dafd184dc2b2c820e587208f6f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting dd5868b996cf4d8893ee7f07a225da36, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:53:50,204 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 1afe536efa8b4685a898b1b2816befd4_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:53:50,216 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:50,218 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:53:50,224 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122803d39e80496043b3a8255a0a845c21ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122803d39e80496043b3a8255a0a845c21ed 2023-02-09 14:53:50,225 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318f19ec1e2a5a4b538c8fbd82109c2ef1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512318f19ec1e2a5a4b538c8fbd82109c2ef1 2023-02-09 14:53:50,227 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b4b759d182fa46c98c4d4a19ed6851ad to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101b4b759d182fa46c98c4d4a19ed6851ad 2023-02-09 14:53:50,229 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010343b2a515be50488c8e443739b5d41536 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010343b2a515be50488c8e443739b5d41536 2023-02-09 14:53:50,229 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-09 14:53:50,229 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:50,258 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 482df8fbf041dd5fc164a0c315c9e3ed#family1#compaction#10 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:53:51,088 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/8166968625444bdb9bdd9e157bcb578b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/8166968625444bdb9bdd9e157bcb578b 2023-02-09 14:53:51,106 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 482df8fbf041dd5fc164a0c315c9e3ed/family1 of 482df8fbf041dd5fc164a0c315c9e3ed into 8166968625444bdb9bdd9e157bcb578b(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:53:51,107 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:51,109 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1675954430201; duration=0sec 2023-02-09 14:53:51,110 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:53:51,120 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120534489afef4534a2b9c45d178d73cecb0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120534489afef4534a2b9c45d178d73cecb0 2023-02-09 14:53:51,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:51,141 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:51,141 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:51,159 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_] to archive 2023-02-09 14:53:51,160 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:53:51,161 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5a4dcd38a1d4499db3905f488131e65f first=Optional[r00] last=Optional[r2] 2023-02-09 14:53:51,162 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5a4dcd38a1d4499db3905f488131e65f}] 2023-02-09 14:53:51,163 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/49c4d4ee2f99488481023cd689def5a5 2023-02-09 14:53:51,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5a4dcd38a1d4499db3905f488131e65f for inclusion in 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:51,166 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ace9169066374876af3f1ddc9d7f0364 2023-02-09 14:53:51,168 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/e99bbaa811034bd9b13db57b307660ff 2023-02-09 14:53:51,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 14:53:51,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/5fafbd5707184371ba6d07cb79be75ff 2023-02-09 14:53:51,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:53:51,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:51,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5a4dcd38a1d4499db3905f488131e65f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fopmvesqoli5rfpge32p24hv182gq39p9suu6q630hfaes755vc3ofkid0r9ef74/family1/5a4dcd38a1d4499db3905f488131e65f 2023-02-09 14:53:51,171 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/728ef16b6dbe45519f1721adbe4a6e1c 2023-02-09 14:53:51,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/635fe7e86951412089176d4f847c0441 2023-02-09 14:53:51,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fopmvesqoli5rfpge32p24hv182gq39p9suu6q630hfaes755vc3ofkid0r9ef74/family1/5a4dcd38a1d4499db3905f488131e65f as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_ 2023-02-09 14:53:51,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/4c0b0796ac0f48d89b7b9819a05c0dd8 2023-02-09 14:53:51,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fopmvesqoli5rfpge32p24hv182gq39p9suu6q630hfaes755vc3ofkid0r9ef74/family1/5a4dcd38a1d4499db3905f488131e65f into 482df8fbf041dd5fc164a0c315c9e3ed/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_ - updating store file list. 2023-02-09 14:53:51,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/f16b37dafd184dc2b2c820e587208f6f 2023-02-09 14:53:51,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/dd5868b996cf4d8893ee7f07a225da36 2023-02-09 14:53:51,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/1afe536efa8b4685a898b1b2816befd4_SeqId_44_ 2023-02-09 14:53:51,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_ into 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:51,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fopmvesqoli5rfpge32p24hv182gq39p9suu6q630hfaes755vc3ofkid0r9ef74/family1/5a4dcd38a1d4499db3905f488131e65f into 482df8fbf041dd5fc164a0c315c9e3ed/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_) 2023-02-09 14:53:51,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fopmvesqoli5rfpge32p24hv182gq39p9suu6q630hfaes755vc3ofkid0r9ef74/family1/5a4dcd38a1d4499db3905f488131e65f 2023-02-09 14:53:51,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:53:51,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:53:51,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 482df8fbf041dd5fc164a0c315c9e3ed/family1 is initiating minor compaction (all files) 2023-02-09 14:53:51,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5c6abd00; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:53:51,196 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 482df8fbf041dd5fc164a0c315c9e3ed/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:51,196 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/8166968625444bdb9bdd9e157bcb578b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp, totalSize=358.6 K 2023-02-09 14:53:51,197 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 2cc17f8f84174aaba112e1ae26bd6191, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:53:51,198 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8166968625444bdb9bdd9e157bcb578b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:53:51,198 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 897a961411a84e7885f917e08e055bd5_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840463 2023-02-09 14:53:51,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:51,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:53:51,202 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130460609ba88d447c6a39b88b199edf078 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130460609ba88d447c6a39b88b199edf078 2023-02-09 14:53:51,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21e16fa3ade444caa960123853c2ce7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21e16fa3ade444caa960123853c2ce7 2023-02-09 14:53:51,206 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057c9ba77c1d504a0d8bcb850facfc2da6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057c9ba77c1d504a0d8bcb850facfc2da6 2023-02-09 14:53:51,206 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-09 14:53:51,206 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 14:53:51,207 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:51,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:51,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:53:51,208 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:51,210 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0260 sec 2023-02-09 14:53:51,243 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 482df8fbf041dd5fc164a0c315c9e3ed#family1#compaction#11 average throughput is 5.70 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:53:51,356 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:51,356 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:51,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 14:53:51,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:51,674 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/.tmp/family1/98569da5a7ef44f8bf1c46d5b1e76975 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/98569da5a7ef44f8bf1c46d5b1e76975 2023-02-09 14:53:51,681 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 482df8fbf041dd5fc164a0c315c9e3ed/family1 of 482df8fbf041dd5fc164a0c315c9e3ed into 98569da5a7ef44f8bf1c46d5b1e76975(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:53:51,681 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:51,681 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1675954431196; duration=0sec 2023-02-09 14:53:51,681 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:53:51,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/8166968625444bdb9bdd9e157bcb578b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_] to archive 2023-02-09 14:53:51,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:53:51,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/2cc17f8f84174aaba112e1ae26bd6191 2023-02-09 14:53:51,749 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/8166968625444bdb9bdd9e157bcb578b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/8166968625444bdb9bdd9e157bcb578b 2023-02-09 14:53:51,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/897a961411a84e7885f917e08e055bd5_SeqId_47_ 2023-02-09 14:53:51,789 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 14:53:51,791 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:51,791 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 14:53:51,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-09 14:53:51,812 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, REOPEN/MOVE}] 2023-02-09 14:53:51,814 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, REOPEN/MOVE 2023-02-09 14:53:51,815 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:51,815 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954431815"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954431815"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954431815"}]},"ts":"1675954431815"} 2023-02-09 14:53:51,818 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:51,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:51,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 482df8fbf041dd5fc164a0c315c9e3ed, disabling compactions & flushes 2023-02-09 14:53:51,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:51,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:51,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. after waiting 0 ms 2023-02-09 14:53:51,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:51,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 14:53:51,984 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:51,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 482df8fbf041dd5fc164a0c315c9e3ed: Waiting for close lock at 1675954431977Running coprocessor pre-close hooks at 1675954431977Disabling compacts and flushes for region at 1675954431977Disabling writes for close at 1675954431977Writing region close event to WAL at 1675954431979 (+2 ms)Running coprocessor post-close hooks at 1675954431983 (+4 ms)Closed at 1675954431984 (+1 ms) 2023-02-09 14:53:51,984 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 482df8fbf041dd5fc164a0c315c9e3ed to self. 2023-02-09 14:53:51,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:51,986 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=CLOSED 2023-02-09 14:53:51,987 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954431986"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954431986"}]},"ts":"1675954431986"} 2023-02-09 14:53:51,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-09 14:53:51,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872 in 170 msec 2023-02-09 14:53:51,991 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=true 2023-02-09 14:53:52,141 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:52,142 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954432141"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954432141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954432141"}]},"ts":"1675954432141"} 2023-02-09 14:53:52,144 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:52,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:52,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 482df8fbf041dd5fc164a0c315c9e3ed, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:52,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:52,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,302 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,304 DEBUG [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:52,304 DEBUG [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:52,304 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 482df8fbf041dd5fc164a0c315c9e3ed columnFamilyName family1 2023-02-09 14:53:52,311 DEBUG [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/98569da5a7ef44f8bf1c46d5b1e76975 2023-02-09 14:53:52,311 INFO [StoreOpener-482df8fbf041dd5fc164a0c315c9e3ed-1] regionserver.HStore(310): Store=482df8fbf041dd5fc164a0c315c9e3ed/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:52,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:53:52,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 482df8fbf041dd5fc164a0c315c9e3ed; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60720112, jitterRate=-0.0951998233795166}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:53:52,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 482df8fbf041dd5fc164a0c315c9e3ed: Running coprocessor pre-open hook at 1675954432301Writing region info on filesystem at 1675954432301Initializing all the Stores at 1675954432302 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675954432302Cleaning up temporary data from old regions at 1675954432314 (+12 ms)Cleaning up detritus from prior splits at 1675954432315 (+1 ms)Running coprocessor post-open hooks at 1675954432318 (+3 ms)Region opened successfully at 1675954432318 2023-02-09 14:53:52,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., pid=17, masterSystemTime=1675954432297 2023-02-09 14:53:52,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:52,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:53:52,321 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=482df8fbf041dd5fc164a0c315c9e3ed, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:52,321 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675954432321"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954432321"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954432321"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954432321"}]},"ts":"1675954432321"} 2023-02-09 14:53:52,325 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-09 14:53:52,325 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 482df8fbf041dd5fc164a0c315c9e3ed, server=jenkins-hbase4.apache.org,43455,1675954416872 in 179 msec 2023-02-09 14:53:52,327 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-09 14:53:52,327 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=482df8fbf041dd5fc164a0c315c9e3ed, REOPEN/MOVE in 513 msec 2023-02-09 14:53:52,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 14:53:52,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 533 msec 2023-02-09 14:53:52,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 966 msec 2023-02-09 14:53:52,335 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:52,335 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:52,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-09 14:53:52,337 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:52,337 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:52,342 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:52,342 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:52,342 INFO [Listener at localhost/33189] compactions.TestMobCompactor(278): alter status finished 2023-02-09 14:53:52,343 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 14:53:52,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:52,346 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:52,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:53:52,347 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955032347 2023-02-09 14:53:52,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:53:52,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 14:53:52,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:52,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x2bd50e12 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:53:52,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35e32542, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:53:52,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:52,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:52,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:52,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:52,355 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:53,189 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112895baf9291bcb45bebdb742590c86dd15 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112895baf9291bcb45bebdb742590c86dd15 2023-02-09 14:53:53,190 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:53,204 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:53:53,205 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51616, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:53:53,212 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,212 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,213 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:53:53,214 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42596, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:53:53,224 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be6affa7c3a54b709a5f89ae09d73620 first=Optional[r05] last=Optional[r8] 2023-02-09 14:53:53,225 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be6affa7c3a54b709a5f89ae09d73620}] 2023-02-09 14:53:53,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be6affa7c3a54b709a5f89ae09d73620 for inclusion in 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:53,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 14:53:53,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:53:53,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 482df8fbf041dd5fc164a0c315c9e3ed: 2023-02-09 14:53:53,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/be6affa7c3a54b709a5f89ae09d73620 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tgg321rhiqaermpeu1tig4gcloqmfoa81s8ka6iko400h9neqtkhu73micslmj4d/family1/be6affa7c3a54b709a5f89ae09d73620 2023-02-09 14:53:53,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tgg321rhiqaermpeu1tig4gcloqmfoa81s8ka6iko400h9neqtkhu73micslmj4d/family1/be6affa7c3a54b709a5f89ae09d73620 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ee9f2252ab7e4e7dace2497fff7d6522_SeqId_53_ 2023-02-09 14:53:53,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tgg321rhiqaermpeu1tig4gcloqmfoa81s8ka6iko400h9neqtkhu73micslmj4d/family1/be6affa7c3a54b709a5f89ae09d73620 into 482df8fbf041dd5fc164a0c315c9e3ed/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ee9f2252ab7e4e7dace2497fff7d6522_SeqId_53_ - updating store file list. 2023-02-09 14:53:53,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ee9f2252ab7e4e7dace2497fff7d6522_SeqId_53_ into 482df8fbf041dd5fc164a0c315c9e3ed/family1 2023-02-09 14:53:53,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tgg321rhiqaermpeu1tig4gcloqmfoa81s8ka6iko400h9neqtkhu73micslmj4d/family1/be6affa7c3a54b709a5f89ae09d73620 into 482df8fbf041dd5fc164a0c315c9e3ed/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/family1/ee9f2252ab7e4e7dace2497fff7d6522_SeqId_53_) 2023-02-09 14:53:53,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tgg321rhiqaermpeu1tig4gcloqmfoa81s8ka6iko400h9neqtkhu73micslmj4d/family1/be6affa7c3a54b709a5f89ae09d73620 2023-02-09 14:53:53,246 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:53,247 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:53:53,249 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8e9377a86a648738d206d845684ecd5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a8e9377a86a648738d206d845684ecd5 2023-02-09 14:53:53,250 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285a128ad9fc5440e38cba17b0be21a415 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285a128ad9fc5440e38cba17b0be21a415 2023-02-09 14:53:53,250 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-09 14:53:53,250 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 14:53:53,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:53,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:53,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 14:53:53,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 14:53:53,251 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:53:53,252 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 14:53:53,254 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 908 msec 2023-02-09 14:53:53,335 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=247 (was 210) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33850 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33820 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5e2ba899-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33880 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x309344f6-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:43455-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x2bd50e12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/2060967321.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x2bd50e12-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33844 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5e2ba899-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33802 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33950 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x6059c336-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x5e2ba899-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x309344f6-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:34012 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_187332668_15 at /127.0.0.1:33920 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x6059c336-SendThread(127.0.0.1:56347) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33878 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33882 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33830 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33862 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33768 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x6059c336 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/2060967321.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x2bd50e12-SendThread(127.0.0.1:56347) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:33962 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=589 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=326 (was 349), ProcessCount=168 (was 168), AvailableMemoryMB=3968 (was 4096) 2023-02-09 14:53:53,346 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=589, MaxFileDescriptor=60000, SystemLoadAverage=326, ProcessCount=168, AvailableMemoryMB=3967 2023-02-09 14:53:53,347 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,347 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:53,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 14:53:53,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:53,351 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:53:53,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-09 14:53:53,352 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,352 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:53,352 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:53:53,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:53,353 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:53,354 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2 empty. 2023-02-09 14:53:53,355 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:53,355 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-09 14:53:53,453 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,453 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:53,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:53,654 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,654 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:53,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:53,764 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 14:53:53,766 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => bea37efa3b23a35293273de94eb1d3d2, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:53:53,956 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:53,956 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:53,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:54,172 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:54,173 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing bea37efa3b23a35293273de94eb1d3d2, disabling compactions & flushes 2023-02-09 14:53:54,173 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,173 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,173 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. after waiting 0 ms 2023-02-09 14:53:54,173 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,173 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,173 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for bea37efa3b23a35293273de94eb1d3d2: Waiting for close lock at 1675954434173Disabling compacts and flushes for region at 1675954434173Disabling writes for close at 1675954434173Writing region close event to WAL at 1675954434173Closed at 1675954434173 2023-02-09 14:53:54,176 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:53:54,177 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954434177"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954434177"}]},"ts":"1675954434177"} 2023-02-09 14:53:54,179 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:53:54,180 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:53:54,180 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954434180"}]},"ts":"1675954434180"} 2023-02-09 14:53:54,182 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 14:53:54,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bea37efa3b23a35293273de94eb1d3d2, ASSIGN}] 2023-02-09 14:53:54,285 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bea37efa3b23a35293273de94eb1d3d2, ASSIGN 2023-02-09 14:53:54,285 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bea37efa3b23a35293273de94eb1d3d2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:53:54,436 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=bea37efa3b23a35293273de94eb1d3d2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:54,437 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954434436"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954434436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954434436"}]},"ts":"1675954434436"} 2023-02-09 14:53:54,439 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure bea37efa3b23a35293273de94eb1d3d2, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:53:54,457 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:54,457 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:54,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:54,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bea37efa3b23a35293273de94eb1d3d2, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:53:54,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:53:54,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,596 INFO [StoreOpener-bea37efa3b23a35293273de94eb1d3d2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,598 DEBUG [StoreOpener-bea37efa3b23a35293273de94eb1d3d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:53:54,598 DEBUG [StoreOpener-bea37efa3b23a35293273de94eb1d3d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:53:54,598 INFO [StoreOpener-bea37efa3b23a35293273de94eb1d3d2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bea37efa3b23a35293273de94eb1d3d2 columnFamilyName family1 2023-02-09 14:53:54,600 INFO [StoreOpener-bea37efa3b23a35293273de94eb1d3d2-1] regionserver.HStore(310): Store=bea37efa3b23a35293273de94eb1d3d2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:53:54,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:53:54,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:53:54,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bea37efa3b23a35293273de94eb1d3d2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59278436, jitterRate=-0.11668246984481812}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:53:54,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bea37efa3b23a35293273de94eb1d3d2: Running coprocessor pre-open hook at 1675954434595Writing region info on filesystem at 1675954434595Initializing all the Stores at 1675954434596 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675954434596Cleaning up temporary data from old regions at 1675954434602 (+6 ms)Cleaning up detritus from prior splits at 1675954434602Running coprocessor post-open hooks at 1675954434607 (+5 ms)Region opened successfully at 1675954434607 2023-02-09 14:53:54,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., pid=21, masterSystemTime=1675954434591 2023-02-09 14:53:54,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:54,610 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=bea37efa3b23a35293273de94eb1d3d2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:54,611 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954434610"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954434610"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954434610"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954434610"}]},"ts":"1675954434610"} 2023-02-09 14:53:54,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-09 14:53:54,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure bea37efa3b23a35293273de94eb1d3d2, server=jenkins-hbase4.apache.org,43455,1675954416872 in 173 msec 2023-02-09 14:53:54,617 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-09 14:53:54,617 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bea37efa3b23a35293273de94eb1d3d2, ASSIGN in 332 msec 2023-02-09 14:53:54,618 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:53:54,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954434618"}]},"ts":"1675954434618"} 2023-02-09 14:53:54,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 14:53:54,653 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:53:54,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.3050 sec 2023-02-09 14:53:55,458 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:55,459 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:55,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:53:55,460 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-09 14:53:55,498 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:55,498 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:55,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:55,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,501 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:55,501 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:55,502 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:55,502 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:55,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,544 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:55,544 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:55,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:55,544 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:55,544 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,545 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:55,545 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:55,545 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:55,546 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:55,546 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:55,546 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:55,546 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:55,546 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:55,547 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:53:55,968 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a11d65237ff54394a661dc76cb8a5341 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a11d65237ff54394a661dc76cb8a5341 2023-02-09 14:53:56,372 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/3157b471bac64a65bac907a1e94d5556 2023-02-09 14:53:56,379 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/3157b471bac64a65bac907a1e94d5556 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556 2023-02-09 14:53:56,385 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 14:53:56,386 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 839ms, sequenceid=5, compaction requested=false 2023-02-09 14:53:56,386 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:56,387 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:56,387 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:56,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:56,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:56,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:56,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,389 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,390 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,390 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:56,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:56,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a3c5394[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:56,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:56,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,392 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:56,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:56,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:56,393 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,393 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:56,394 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:56,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:56,397 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:56,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:56,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:56,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,397 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:56,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,400 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,401 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:56,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:56,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:56,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,404 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:56,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,405 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:56,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:56,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:56,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,405 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:56,505 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:56,505 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:56,505 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:56,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:56,511 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:56,511 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:56,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:56,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:56,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,514 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:56,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:56,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:56,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:56,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,516 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,516 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:56,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:56,516 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,516 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:56,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:56,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:56,517 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:56,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:56,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:56,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:56,518 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:56,518 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:56,518 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:53:56,933 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b87d87acdf214b82aa41a98126f6cbc4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b87d87acdf214b82aa41a98126f6cbc4 2023-02-09 14:53:56,939 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/9c536ad0667c42a1bf7c7d563a0e93aa 2023-02-09 14:53:56,945 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/9c536ad0667c42a1bf7c7d563a0e93aa as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa 2023-02-09 14:53:56,950 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:53:56,951 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 433ms, sequenceid=9, compaction requested=false 2023-02-09 14:53:56,952 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:56,952 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:56,954 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,954 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,954 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,954 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:56,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18c9ba3d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:56,955 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:56,955 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:56,958 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,958 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:56,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:56,959 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:56,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:56,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:56,962 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:56,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:56,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:56,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,963 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:56,965 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,965 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:56,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,965 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:56,965 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:56,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:56,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,967 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:56,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:56,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:56,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:56,978 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:56,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:56,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:56,979 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,979 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,979 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:56,979 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:56,979 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:56,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:56,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:57,079 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:57,079 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:57,080 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:57,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:57,085 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:57,085 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:57,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:57,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:57,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,088 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:57,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:57,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:57,088 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:57,089 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,089 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,090 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:57,090 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:57,091 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,091 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,092 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,092 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:57,092 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:57,092 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:57,093 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:57,093 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:57,093 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:57,093 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:57,093 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:57,508 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:53:57,509 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120154754cd79b3148deb90932dc8b304455 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120154754cd79b3148deb90932dc8b304455 2023-02-09 14:53:57,919 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/4e34cb02e8c94936b245b47dfd65dd6b 2023-02-09 14:53:57,926 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/4e34cb02e8c94936b245b47dfd65dd6b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b 2023-02-09 14:53:57,932 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 14:53:57,933 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 840ms, sequenceid=13, compaction requested=true 2023-02-09 14:53:57,934 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:57,934 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:57,936 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,936 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:57,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:57,936 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:57,936 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:57,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:57,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:57,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:57,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41f089b0[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:57,944 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:57,944 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,946 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:57,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:57,946 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,946 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:57,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:57,948 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,948 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:57,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:57,948 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:57,948 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:57,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:57,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:57,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:57,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:57,951 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:57,951 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:57,951 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:57,951 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,951 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:57,953 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,953 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,953 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:57,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,953 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:57,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:57,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:57,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:57,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,955 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:57,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,957 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:57,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:57,958 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:57,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:57,958 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:57,957 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:57,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:57,958 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:57,958 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:57,959 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:57,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:57,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:57,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:53:58,059 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:58,059 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,059 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:58,066 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,066 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:58,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:58,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,072 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:58,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:58,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,074 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,075 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:58,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:58,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,077 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:58,080 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:58,080 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:58,080 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,080 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:58,080 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:58,498 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120511d6b228352a489eb503d501945907d6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120511d6b228352a489eb503d501945907d6 2023-02-09 14:53:58,506 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/447f19b96c1c41cf9284313d69e76497 2023-02-09 14:53:58,512 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/447f19b96c1c41cf9284313d69e76497 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497 2023-02-09 14:53:58,518 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 14:53:58,519 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 439ms, sequenceid=17, compaction requested=true 2023-02-09 14:53:58,519 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:58,519 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:58,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:58,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,520 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:58,520 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:58,522 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,522 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,522 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,522 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:58,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:58,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d314aa5[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:58,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:58,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,525 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,525 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:58,525 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:58,525 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:58,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:58,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:58,527 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:58,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:58,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:58,530 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:58,530 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:58,530 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:58,530 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,530 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:58,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,533 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,536 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:58,539 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,539 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,540 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,540 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:58,540 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:58,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,540 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,540 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,640 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:58,641 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,641 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:58,647 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,647 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:58,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:58,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,650 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,650 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,651 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:58,651 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:58,651 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,651 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,653 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,653 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,653 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,653 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,655 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:58,655 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:58,656 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:58,656 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,656 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:58,656 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:58,677 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283838b39173a746ae832e1ecad0167a64 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283838b39173a746ae832e1ecad0167a64 2023-02-09 14:53:58,685 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/1bd5e679a2084b8caa69598939e15530 2023-02-09 14:53:58,692 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/1bd5e679a2084b8caa69598939e15530 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530 2023-02-09 14:53:58,697 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 14:53:58,698 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 42ms, sequenceid=21, compaction requested=true 2023-02-09 14:53:58,698 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:58,698 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,698 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:58,698 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:58,698 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,699 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:58,699 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:58,700 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,700 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,701 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,701 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:58,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35f90264[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:58,702 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:58,702 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,703 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,703 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:58,703 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:58,703 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:58,703 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,704 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:58,706 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:58,706 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,706 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,706 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:58,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:58,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:58,709 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:58,710 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:58,710 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:58,710 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,710 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:58,714 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,714 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,714 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,715 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,716 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,718 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,718 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,718 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:53:58,718 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:58,718 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,719 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,719 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,719 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:58,719 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,720 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:58,720 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,720 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,820 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:58,820 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,820 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:58,825 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,825 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:58,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:58,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,828 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,828 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,829 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:58,829 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:58,829 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,829 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,832 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,832 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,832 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,832 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:58,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:58,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:58,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:58,834 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:58,834 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,835 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:58,835 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:58,849 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231948acacf7d3f45ef877a91dea2f73226 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231948acacf7d3f45ef877a91dea2f73226 2023-02-09 14:53:58,855 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/ca22817aa4c347c3b0a265a950f68152 2023-02-09 14:53:58,861 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/ca22817aa4c347c3b0a265a950f68152 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152 2023-02-09 14:53:58,867 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 14:53:58,868 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 32ms, sequenceid=25, compaction requested=true 2023-02-09 14:53:58,868 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:58,868 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:58,868 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:58,868 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:58,868 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,868 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:58,868 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:58,870 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:58,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60b45eb4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:58,872 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:58,872 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,873 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,873 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:58,873 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,873 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:58,873 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:58,873 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:58,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:58,875 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,875 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,875 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:58,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:58,878 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:58,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:58,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:58,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,878 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:58,881 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,881 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,881 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:58,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,882 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:58,883 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,886 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:58,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:58,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:53:58,887 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,887 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:58,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:58,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:58,887 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,888 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:58,888 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:58,988 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:58,988 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,988 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:58,993 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:58,993 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:58,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:58,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:58,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,996 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:58,996 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:58,997 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:58,997 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:58,997 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,997 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,998 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,998 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:58,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:58,999 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:58,999 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:58,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:59,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:59,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,000 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:59,000 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:59,001 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:59,001 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:59,001 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:59,001 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:59,001 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:59,001 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:53:59,418 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010173dea17576224aaa8c06c675dff3e49e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010173dea17576224aaa8c06c675dff3e49e 2023-02-09 14:53:59,830 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/c0b8863d74b04177a0efdbefd5371ff9 2023-02-09 14:53:59,838 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/c0b8863d74b04177a0efdbefd5371ff9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9 2023-02-09 14:53:59,843 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 14:53:59,845 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 844ms, sequenceid=29, compaction requested=true 2023-02-09 14:53:59,845 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:53:59,845 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:53:59,847 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,847 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:59,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:59,848 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,848 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:59,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:53:59,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@254cf61b[Count = 0] remaining members to acquire global barrier 2023-02-09 14:53:59,850 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:53:59,850 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,852 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:53:59,853 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,853 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:53:59,855 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:53:59,855 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:59,855 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,855 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:59,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:53:59,858 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:53:59,859 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:53:59,859 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:53:59,859 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,859 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:53:59,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:53:59,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:59,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:53:59,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:53:59,864 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,866 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:59,866 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:59,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:59,867 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,866 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:53:59,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:59,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:53:59,866 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:53:59,867 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,867 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,868 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,868 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:53:59,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:53:59,868 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,868 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:53:59,868 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:53:59,968 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:53:59,968 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:59,968 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:59,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:53:59,975 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:53:59,976 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:53:59,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:53:59,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:53:59,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,978 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:53:59,978 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:59,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:53:59,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:53:59,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:53:59,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:53:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:53:59,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:53:59,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:53:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:53:59,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:53:59,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:53:59,985 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:53:59,985 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:53:59,985 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:53:59,985 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:53:59,985 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:53:59,985 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:53:59,986 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:53:59,986 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:00,408 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031447c3a6479a437f933a86c144e8ca02 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031447c3a6479a437f933a86c144e8ca02 2023-02-09 14:54:00,819 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/ca3fc377c77443328ab9ccf3833f7ab6 2023-02-09 14:54:00,828 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/ca3fc377c77443328ab9ccf3833f7ab6 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6 2023-02-09 14:54:00,835 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 14:54:00,836 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 850ms, sequenceid=33, compaction requested=true 2023-02-09 14:54:00,837 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:00,837 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:00,837 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:00,837 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:00,837 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,837 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:54:00,837 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:54:00,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:00,839 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:00,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:00,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:00,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:00,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:54:00,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@727fe475[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:00,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:00,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,843 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:00,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,844 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:00,848 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,848 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:54:00,848 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:00,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:00,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:00,849 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:54:00,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:00,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:00,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:00,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:00,853 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:00,853 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:54:00,853 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:00,853 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:00,853 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,853 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:00,855 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,855 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,855 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:00,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:00,856 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:00,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,857 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:00,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:00,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:54:00,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:00,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:00,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:54:00,860 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:00,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,860 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:00,861 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,861 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:00,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:00,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:00,861 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,861 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:00,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,962 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:00,962 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:00,962 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:00,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:00,967 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:00,967 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:00,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:00,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:00,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,971 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:54:00,971 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:00,971 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:54:00,971 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:00,972 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,972 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,974 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:00,974 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:00,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:00,974 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:00,974 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:00,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,975 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,975 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:54:00,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:00,975 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:54:00,975 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:00,975 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:54:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:00,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:00,976 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:00,976 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:54:00,976 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:01,397 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076e0e6c2ed5b6480aa394099703be999e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509076e0e6c2ed5b6480aa394099703be999e 2023-02-09 14:54:01,804 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/80fa94de04f24cf19a9fa515e01b6d08 2023-02-09 14:54:01,809 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/80fa94de04f24cf19a9fa515e01b6d08 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08 2023-02-09 14:54:01,814 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 14:54:01,816 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 840ms, sequenceid=37, compaction requested=true 2023-02-09 14:54:01,816 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:01,816 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:01,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:01,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:01,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:54:01,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:54:01,817 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,817 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:01,818 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,818 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:01,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:01,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:54:01,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fe8327c[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:01,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:01,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,820 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,820 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:01,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:54:01,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:01,821 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,821 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:01,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,822 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:54:01,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:01,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:01,822 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:01,822 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:54:01,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:01,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:01,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:01,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:01,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:01,825 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:54:01,825 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:01,825 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:01,825 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,825 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:01,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:01,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:01,828 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,829 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:01,831 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:01,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:54:01,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:01,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:01,832 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,832 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,832 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:01,833 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:01,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:01,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:01,933 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:01,933 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:01,933 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:01,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:01,938 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:01,938 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:01,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:01,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:01,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,941 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:54:01,941 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:01,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:54:01,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:01,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,943 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:01,943 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:01,944 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:01,944 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:01,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:54:01,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:54:01,945 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:54:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:01,946 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:01,947 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:01,947 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. started... 2023-02-09 14:54:01,947 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing bea37efa3b23a35293273de94eb1d3d2 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:02,362 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a7547cdbf3b7413b89f157998871e388 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7547cdbf3b7413b89f157998871e388 2023-02-09 14:54:02,768 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/a01ce7412d1e44c08ab81da79a5536e8 2023-02-09 14:54:02,775 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/a01ce7412d1e44c08ab81da79a5536e8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8 2023-02-09 14:54:02,780 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 14:54:02,781 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bea37efa3b23a35293273de94eb1d3d2 in 834ms, sequenceid=41, compaction requested=true 2023-02-09 14:54:02,781 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:02,781 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:02,781 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:02,781 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:02,781 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,781 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:54:02,782 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:54:02,783 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,783 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:02,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:02,784 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,784 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:02,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:54:02,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4deb9ccb[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:02,785 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:02,785 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,788 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,788 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:02,788 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:54:02,788 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,788 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:02,788 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:02,789 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,789 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:54:02,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:02,790 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:02,790 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:54:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:02,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:02,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:02,792 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:54:02,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:02,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:02,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,792 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:02,794 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,794 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,794 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:02,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:02,794 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:02,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:02,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,796 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:02,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:02,798 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:02,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:02,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 14:54:02,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:02,798 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:02,799 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,799 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:02,799 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,800 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:02,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:02,800 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:02,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:02,900 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:02,900 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:02,900 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:02,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:02,902 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-09 14:54:02,904 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:02,905 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:02,906 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:02,906 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955042906 2023-02-09 14:54:02,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:02,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 14:54:02,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:02,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x06468800 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:02,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a2374bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:02,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:03,748 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010399df52e9d3654d98a05c27fb99e1a0dc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010399df52e9d3654d98a05c27fb99e1a0dc 2023-02-09 14:54:03,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:03,765 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:03,766 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32922, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:03,774 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:03,774 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:03,774 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:03,775 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59452, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:03,789 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6b826df710224ecdb9707c0ecb229f89 first=Optional[r5] last=Optional[r6] 2023-02-09 14:54:03,790 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6b826df710224ecdb9707c0ecb229f89}] 2023-02-09 14:54:03,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6b826df710224ecdb9707c0ecb229f89 for inclusion in bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:03,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 14:54:03,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:03,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:03,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/6b826df710224ecdb9707c0ecb229f89 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__mu9qncf6f2ocqklbh2bu23oibkf2rs58o6qr94f13gn400j157kt94v1l9g9pvjf/family1/6b826df710224ecdb9707c0ecb229f89 2023-02-09 14:54:03,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__mu9qncf6f2ocqklbh2bu23oibkf2rs58o6qr94f13gn400j157kt94v1l9g9pvjf/family1/6b826df710224ecdb9707c0ecb229f89 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_ 2023-02-09 14:54:03,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__mu9qncf6f2ocqklbh2bu23oibkf2rs58o6qr94f13gn400j157kt94v1l9g9pvjf/family1/6b826df710224ecdb9707c0ecb229f89 into bea37efa3b23a35293273de94eb1d3d2/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_ - updating store file list. 2023-02-09 14:54:03,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_ into bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:03,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__mu9qncf6f2ocqklbh2bu23oibkf2rs58o6qr94f13gn400j157kt94v1l9g9pvjf/family1/6b826df710224ecdb9707c0ecb229f89 into bea37efa3b23a35293273de94eb1d3d2/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_) 2023-02-09 14:54:03,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__mu9qncf6f2ocqklbh2bu23oibkf2rs58o6qr94f13gn400j157kt94v1l9g9pvjf/family1/6b826df710224ecdb9707c0ecb229f89 2023-02-09 14:54:03,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:54:03,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:54:03,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): bea37efa3b23a35293273de94eb1d3d2/family1 is initiating minor compaction 2023-02-09 14:54:03,813 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bea37efa3b23a35293273de94eb1d3d2/family1 in testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:03,814 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp, totalSize=49.7 K 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 9c536ad0667c42a1bf7c7d563a0e93aa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4e34cb02e8c94936b245b47dfd65dd6b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 447f19b96c1c41cf9284313d69e76497, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 1bd5e679a2084b8caa69598939e15530, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting ca22817aa4c347c3b0a265a950f68152, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:54:03,814 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting c0b8863d74b04177a0efdbefd5371ff9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:54:03,815 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting ca3fc377c77443328ab9ccf3833f7ab6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:54:03,815 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 80fa94de04f24cf19a9fa515e01b6d08, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:54:03,815 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting a01ce7412d1e44c08ab81da79a5536e8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:54:03,815 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:54:03,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@20c27cea; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:03,817 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:03,819 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:03,821 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010173dea17576224aaa8c06c675dff3e49e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010173dea17576224aaa8c06c675dff3e49e 2023-02-09 14:54:03,822 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031447c3a6479a437f933a86c144e8ca02 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601031447c3a6479a437f933a86c144e8ca02 2023-02-09 14:54:03,822 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 14:54:03,822 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 14:54:03,823 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:03,823 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:03,888 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): bea37efa3b23a35293273de94eb1d3d2#family1#compaction#22 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:04,306 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231340a545bbdf1434e9bb0716cd6363291 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231340a545bbdf1434e9bb0716cd6363291 2023-02-09 14:54:04,311 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/91737be5569d4f5288ae706d6cc4209f as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/91737be5569d4f5288ae706d6cc4209f 2023-02-09 14:54:04,313 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:04,325 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in bea37efa3b23a35293273de94eb1d3d2/family1 of bea37efa3b23a35293273de94eb1d3d2 into 91737be5569d4f5288ae706d6cc4209f(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:04,325 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:04,325 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675954443813; duration=0sec 2023-02-09 14:54:04,325 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:04,340 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:04,341 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:04,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_] to archive 2023-02-09 14:54:04,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:04,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/9c536ad0667c42a1bf7c7d563a0e93aa 2023-02-09 14:54:04,359 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/4e34cb02e8c94936b245b47dfd65dd6b 2023-02-09 14:54:04,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/447f19b96c1c41cf9284313d69e76497 2023-02-09 14:54:04,363 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/517d41851adb4852bd204796426f77aa first=Optional[r1] last=Optional[r4] 2023-02-09 14:54:04,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/1bd5e679a2084b8caa69598939e15530 2023-02-09 14:54:04,364 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/517d41851adb4852bd204796426f77aa}] 2023-02-09 14:54:04,366 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca22817aa4c347c3b0a265a950f68152 2023-02-09 14:54:04,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/517d41851adb4852bd204796426f77aa for inclusion in bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:04,367 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/c0b8863d74b04177a0efdbefd5371ff9 2023-02-09 14:54:04,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/ca3fc377c77443328ab9ccf3833f7ab6 2023-02-09 14:54:04,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/80fa94de04f24cf19a9fa515e01b6d08 2023-02-09 14:54:04,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 14:54:04,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:04,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:04,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/517d41851adb4852bd204796426f77aa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__emntamh6sh3p9qpk2bal6r75b63pd9i2h145tu0v0r38pkord2uce2dv4hkg4ueo/family1/517d41851adb4852bd204796426f77aa 2023-02-09 14:54:04,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/a01ce7412d1e44c08ab81da79a5536e8 2023-02-09 14:54:04,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__emntamh6sh3p9qpk2bal6r75b63pd9i2h145tu0v0r38pkord2uce2dv4hkg4ueo/family1/517d41851adb4852bd204796426f77aa as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_ 2023-02-09 14:54:04,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__emntamh6sh3p9qpk2bal6r75b63pd9i2h145tu0v0r38pkord2uce2dv4hkg4ueo/family1/517d41851adb4852bd204796426f77aa into bea37efa3b23a35293273de94eb1d3d2/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_ - updating store file list. 2023-02-09 14:54:04,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/e5ab15c043b84abea2963c1a6b7fd3b7_SeqId_44_ 2023-02-09 14:54:04,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_ into bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:04,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__emntamh6sh3p9qpk2bal6r75b63pd9i2h145tu0v0r38pkord2uce2dv4hkg4ueo/family1/517d41851adb4852bd204796426f77aa into bea37efa3b23a35293273de94eb1d3d2/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_) 2023-02-09 14:54:04,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__emntamh6sh3p9qpk2bal6r75b63pd9i2h145tu0v0r38pkord2uce2dv4hkg4ueo/family1/517d41851adb4852bd204796426f77aa 2023-02-09 14:54:04,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:54:04,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:54:04,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): bea37efa3b23a35293273de94eb1d3d2/family1 is initiating minor compaction (all files) 2023-02-09 14:54:04,385 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bea37efa3b23a35293273de94eb1d3d2/family1 in testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:54:04,386 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/91737be5569d4f5288ae706d6cc4209f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp, totalSize=178.1 K 2023-02-09 14:54:04,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@167744ae; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:04,386 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 3157b471bac64a65bac907a1e94d5556, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:54:04,387 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 91737be5569d4f5288ae706d6cc4209f, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:54:04,388 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 86894c91e35548e8a275e8bb6088b4c0_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240463 2023-02-09 14:54:04,389 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:04,390 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:04,393 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120154754cd79b3148deb90932dc8b304455 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120154754cd79b3148deb90932dc8b304455 2023-02-09 14:54:04,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120511d6b228352a489eb503d501945907d6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120511d6b228352a489eb503d501945907d6 2023-02-09 14:54:04,397 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283838b39173a746ae832e1ecad0167a64 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512283838b39173a746ae832e1ecad0167a64 2023-02-09 14:54:04,398 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231948acacf7d3f45ef877a91dea2f73226 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231948acacf7d3f45ef877a91dea2f73226 2023-02-09 14:54:04,398 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 14:54:04,398 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 14:54:04,398 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:04,398 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:04,410 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): bea37efa3b23a35293273de94eb1d3d2#family1#compaction#23 average throughput is 15.56 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:04,495 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113033ae6026925d4dba95211d0c2c89e50b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113033ae6026925d4dba95211d0c2c89e50b 2023-02-09 14:54:04,496 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:04,518 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:04,518 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:04,532 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a26560f86fe472dadfff5b60be7cd5c first=Optional[r00] last=Optional[r8] 2023-02-09 14:54:04,533 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a26560f86fe472dadfff5b60be7cd5c}] 2023-02-09 14:54:04,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a26560f86fe472dadfff5b60be7cd5c for inclusion in bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:04,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 14:54:04,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:04,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:04,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a26560f86fe472dadfff5b60be7cd5c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hh1c2krf39p5d1mofc7vim2o1r5a7i842ap9eg37gd379s43b28hn8gmt9n9att4/family1/6a26560f86fe472dadfff5b60be7cd5c 2023-02-09 14:54:04,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hh1c2krf39p5d1mofc7vim2o1r5a7i842ap9eg37gd379s43b28hn8gmt9n9att4/family1/6a26560f86fe472dadfff5b60be7cd5c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/aaafc90f1b1a45b9b87c394e673b9bca_SeqId_49_ 2023-02-09 14:54:04,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hh1c2krf39p5d1mofc7vim2o1r5a7i842ap9eg37gd379s43b28hn8gmt9n9att4/family1/6a26560f86fe472dadfff5b60be7cd5c into bea37efa3b23a35293273de94eb1d3d2/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/aaafc90f1b1a45b9b87c394e673b9bca_SeqId_49_ - updating store file list. 2023-02-09 14:54:04,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/aaafc90f1b1a45b9b87c394e673b9bca_SeqId_49_ into bea37efa3b23a35293273de94eb1d3d2/family1 2023-02-09 14:54:04,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hh1c2krf39p5d1mofc7vim2o1r5a7i842ap9eg37gd379s43b28hn8gmt9n9att4/family1/6a26560f86fe472dadfff5b60be7cd5c into bea37efa3b23a35293273de94eb1d3d2/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/aaafc90f1b1a45b9b87c394e673b9bca_SeqId_49_) 2023-02-09 14:54:04,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hh1c2krf39p5d1mofc7vim2o1r5a7i842ap9eg37gd379s43b28hn8gmt9n9att4/family1/6a26560f86fe472dadfff5b60be7cd5c 2023-02-09 14:54:04,555 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:04,556 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:04,558 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7547cdbf3b7413b89f157998871e388 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7547cdbf3b7413b89f157998871e388 2023-02-09 14:54:04,560 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b87d87acdf214b82aa41a98126f6cbc4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b87d87acdf214b82aa41a98126f6cbc4 2023-02-09 14:54:04,562 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a11d65237ff54394a661dc76cb8a5341 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a11d65237ff54394a661dc76cb8a5341 2023-02-09 14:54:04,562 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 14:54:04,562 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 14:54:04,563 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:04,563 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:04,563 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:54:04,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:54:04,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:04,564 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:04,567 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.6610 sec 2023-02-09 14:54:04,720 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=261 (was 247) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/3af5b042e35b4585a570dc412f3f2335 block BP-477111177-172.31.14.131-1675954412651:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:37034 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x23c84b95-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56990 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-477111177-172.31.14.131-1675954412651:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-477111177-172.31.14.131-1675954412651:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231cce311e25bd2471bae8c6d309c104e03 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x06468800-SendThread(127.0.0.1:56347) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:37026 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:37014 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56978 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56948 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56942 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:36960 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56968 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:36972 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x06468800 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/2060967321.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x29cf6e02-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56940 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:37002 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_187332668_15 at /127.0.0.1:57062 [Receiving block BP-477111177-172.31.14.131-1675954412651:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:56347@0x06468800-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_187332668_15 at /127.0.0.1:56934 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56974 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:57058 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-557271950_15 at /127.0.0.1:56958 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=612 (was 589) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=291 (was 326), ProcessCount=168 (was 168), AvailableMemoryMB=3652 (was 3967) 2023-02-09 14:54:04,736 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=260, OpenFileDescriptor=612, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=168, AvailableMemoryMB=3651 2023-02-09 14:54:04,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:04,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:04,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:54:04,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:04,742 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:04,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-09 14:54:04,743 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:04,743 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:04,743 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:04,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 14:54:04,745 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:04,746 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c empty. 2023-02-09 14:54:04,747 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:04,747 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-09 14:54:04,759 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:04,761 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9a6c61f775706f7023a02aced32af95c, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 9a6c61f775706f7023a02aced32af95c, disabling compactions & flushes 2023-02-09 14:54:04,770 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. after waiting 0 ms 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:04,770 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:04,770 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 9a6c61f775706f7023a02aced32af95c: Waiting for close lock at 1675954444770Disabling compacts and flushes for region at 1675954444770Disabling writes for close at 1675954444770Writing region close event to WAL at 1675954444770Closed at 1675954444770 2023-02-09 14:54:04,773 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:04,774 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954444773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954444773"}]},"ts":"1675954444773"} 2023-02-09 14:54:04,775 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:04,776 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:04,776 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954444776"}]},"ts":"1675954444776"} 2023-02-09 14:54:04,778 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 14:54:04,784 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=9a6c61f775706f7023a02aced32af95c, ASSIGN}] 2023-02-09 14:54:04,786 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=9a6c61f775706f7023a02aced32af95c, ASSIGN 2023-02-09 14:54:04,786 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=9a6c61f775706f7023a02aced32af95c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:04,844 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:04,845 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:04,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 14:54:04,938 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=9a6c61f775706f7023a02aced32af95c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:04,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954444937"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954444937"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954444937"}]},"ts":"1675954444937"} 2023-02-09 14:54:04,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 9a6c61f775706f7023a02aced32af95c, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:05,046 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:05,046 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:05,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 14:54:05,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a6c61f775706f7023a02aced32af95c, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:05,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:05,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,097 INFO [StoreOpener-9a6c61f775706f7023a02aced32af95c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,099 DEBUG [StoreOpener-9a6c61f775706f7023a02aced32af95c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:05,099 DEBUG [StoreOpener-9a6c61f775706f7023a02aced32af95c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:05,099 INFO [StoreOpener-9a6c61f775706f7023a02aced32af95c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a6c61f775706f7023a02aced32af95c columnFamilyName family1 2023-02-09 14:54:05,101 INFO [StoreOpener-9a6c61f775706f7023a02aced32af95c-1] regionserver.HStore(310): Store=9a6c61f775706f7023a02aced32af95c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:05,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:54:05,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:05,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a6c61f775706f7023a02aced32af95c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61611981, jitterRate=-0.08190993964672089}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:05,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a6c61f775706f7023a02aced32af95c: Running coprocessor pre-open hook at 1675954445096Writing region info on filesystem at 1675954445096Initializing all the Stores at 1675954445097 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954445097Cleaning up temporary data from old regions at 1675954445103 (+6 ms)Cleaning up detritus from prior splits at 1675954445103Running coprocessor post-open hooks at 1675954445108 (+5 ms)Region opened successfully at 1675954445108 2023-02-09 14:54:05,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c., pid=25, masterSystemTime=1675954445092 2023-02-09 14:54:05,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,111 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=9a6c61f775706f7023a02aced32af95c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,111 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954445111"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954445111"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954445111"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954445111"}]},"ts":"1675954445111"} 2023-02-09 14:54:05,114 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-09 14:54:05,114 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 9a6c61f775706f7023a02aced32af95c, server=jenkins-hbase4.apache.org,43455,1675954416872 in 173 msec 2023-02-09 14:54:05,116 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-09 14:54:05,116 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=9a6c61f775706f7023a02aced32af95c, ASSIGN in 330 msec 2023-02-09 14:54:05,117 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:05,117 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954445117"}]},"ts":"1675954445117"} 2023-02-09 14:54:05,119 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 14:54:05,127 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:05,128 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 388 msec 2023-02-09 14:54:05,283 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/.tmp/family1/3af5b042e35b4585a570dc412f3f2335 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3af5b042e35b4585a570dc412f3f2335 2023-02-09 14:54:05,289 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in bea37efa3b23a35293273de94eb1d3d2/family1 of bea37efa3b23a35293273de94eb1d3d2 into 3af5b042e35b4585a570dc412f3f2335(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:05,289 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bea37efa3b23a35293273de94eb1d3d2: 2023-02-09 14:54:05,289 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675954444385; duration=0sec 2023-02-09 14:54:05,289 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:05,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/91737be5569d4f5288ae706d6cc4209f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_] to archive 2023-02-09 14:54:05,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:05,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/3157b471bac64a65bac907a1e94d5556 2023-02-09 14:54:05,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/91737be5569d4f5288ae706d6cc4209f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/91737be5569d4f5288ae706d6cc4209f 2023-02-09 14:54:05,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/family1/86894c91e35548e8a275e8bb6088b4c0_SeqId_47_ 2023-02-09 14:54:05,347 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:05,347 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:05,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 14:54:05,348 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-09 14:54:05,376 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:05,376 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:05,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:05,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,378 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:05,378 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:05,378 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:05,378 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:05,379 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,379 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,380 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:05,380 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:05,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:05,381 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,381 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:05,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,382 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:05,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:05,382 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,382 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:05,383 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:54:05,804 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fc678681ced747ceb21d0d782125ff5b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130fc678681ced747ceb21d0d782125ff5b 2023-02-09 14:54:05,807 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/5091712852db4880850831ec1be78347 2023-02-09 14:54:05,813 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/5091712852db4880850831ec1be78347 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/5091712852db4880850831ec1be78347 2023-02-09 14:54:05,819 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/5091712852db4880850831ec1be78347, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 14:54:05,820 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 437ms, sequenceid=5, compaction requested=false 2023-02-09 14:54:05,820 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:05,821 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:05,821 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:05,823 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,823 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:05,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:05,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:05,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fd55743[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:05,824 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:05,825 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:05,826 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:05,826 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,826 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:05,831 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:05,831 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,832 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:05,832 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:05,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:05,834 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:05,834 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:05,834 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:05,834 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,834 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:05,835 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,835 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,835 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:05,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:05,836 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,837 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:05,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:05,839 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:05,839 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:05,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:05,840 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:05,840 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,840 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:05,840 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:05,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:05,840 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,940 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:05,940 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:05,940 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:05,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:05,946 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:05,946 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:05,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:05,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:05,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,948 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:05,948 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:05,948 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:05,948 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:05,948 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,949 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,950 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:05,950 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:05,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:05,950 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:05,950 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:05,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,951 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:05,952 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:05,952 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:05,952 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:54:06,366 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281186840483184e7fa91b99508e4121ce to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511281186840483184e7fa91b99508e4121ce 2023-02-09 14:54:06,372 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/720e8898e07842869b7ee538ff167059 2023-02-09 14:54:06,378 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/720e8898e07842869b7ee538ff167059 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059 2023-02-09 14:54:06,384 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:54:06,385 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 433ms, sequenceid=9, compaction requested=false 2023-02-09 14:54:06,386 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:06,386 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:06,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:06,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:06,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:06,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:06,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,389 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:06,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:06,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:06,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:06,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2844680[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:06,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:06,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,392 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,392 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:06,392 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:06,392 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:06,392 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,392 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:06,393 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:06,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:06,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:06,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:06,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:06,396 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:06,396 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:06,396 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:06,396 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,396 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:06,398 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,398 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,398 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:06,398 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:06,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:06,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:06,399 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:06,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:06,402 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:06,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:06,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:06,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:06,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:06,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:06,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:06,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,403 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:06,403 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:06,503 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:06,504 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:06,504 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:06,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:06,509 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:06,509 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:06,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,511 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:06,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:06,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:06,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:06,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,513 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:06,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:06,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:06,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:06,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:06,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:06,515 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:06,515 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:06,515 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:06,929 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201aea112472f364971bc8cf6170d4c43ba to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aea112472f364971bc8cf6170d4c43ba 2023-02-09 14:54:07,335 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/dae99a8a2730467593c52a1a739e9ed9 2023-02-09 14:54:07,341 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/dae99a8a2730467593c52a1a739e9ed9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9 2023-02-09 14:54:07,345 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 14:54:07,346 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 831ms, sequenceid=13, compaction requested=true 2023-02-09 14:54:07,347 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:07,347 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:07,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:07,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:07,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:07,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:07,348 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,348 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:07,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c31ef7e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:07,350 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:07,350 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,351 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,351 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:07,351 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:07,352 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,352 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:07,353 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,354 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:07,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:07,356 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:07,356 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:07,356 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:07,356 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,356 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:07,359 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,359 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,359 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,359 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,361 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,365 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,365 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,365 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:07,365 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:07,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:07,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,365 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:07,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,366 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:07,466 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:07,466 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,466 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:07,472 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,472 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:07,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:07,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,475 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:07,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:07,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:07,476 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,476 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,477 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,478 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:07,478 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:07,478 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:07,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:07,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:07,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:07,483 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:07,483 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:07,483 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:07,483 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:07,484 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:07,902 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120592588665927c4c6a86cf50d352abf4d3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120592588665927c4c6a86cf50d352abf4d3 2023-02-09 14:54:08,307 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/58fd78c567c049c69b49573fa9530a36 2023-02-09 14:54:08,313 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/58fd78c567c049c69b49573fa9530a36 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36 2023-02-09 14:54:08,317 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 14:54:08,318 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 834ms, sequenceid=17, compaction requested=true 2023-02-09 14:54:08,318 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:08,318 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:08,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:08,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:08,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:08,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:08,320 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,320 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:08,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5dad3ff7[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:08,322 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:08,322 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,323 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:08,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:08,325 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:08,325 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,326 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:08,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:08,328 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:08,328 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:08,328 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:08,328 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,328 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:08,329 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,329 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,329 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,330 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,331 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,333 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,333 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,333 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,334 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:08,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:08,334 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,434 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:08,434 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,435 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:08,440 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,441 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:08,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:08,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,443 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:08,443 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,443 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:08,443 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:08,444 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,444 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,446 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,446 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,446 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:08,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,446 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,447 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:08,447 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,447 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:08,447 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,447 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:08,447 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:08,447 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:08,447 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:08,448 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:08,448 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:08,466 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d3ecdf3caf644285890137161f6bcee9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3ecdf3caf644285890137161f6bcee9 2023-02-09 14:54:08,873 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/45ec95573fa743f5bafe2112832fe7d5 2023-02-09 14:54:08,879 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/45ec95573fa743f5bafe2112832fe7d5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5 2023-02-09 14:54:08,885 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 14:54:08,886 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 438ms, sequenceid=21, compaction requested=true 2023-02-09 14:54:08,886 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:08,886 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:08,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:08,889 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7867144e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:08,891 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:08,891 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,892 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:08,893 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:08,893 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,893 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:08,895 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:08,895 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,895 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,895 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:08,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:08,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:08,897 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:08,897 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:08,898 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:08,898 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,898 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:08,900 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,900 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,900 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,900 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,902 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,910 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:08,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:08,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:08,911 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:08,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:09,011 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:09,012 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:09,012 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:09,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:09,019 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:09,019 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:09,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:09,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:09,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,026 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:09,026 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:09,027 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:09,027 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:09,028 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,028 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,030 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:09,030 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:09,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:09,031 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,031 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:09,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:09,033 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:09,033 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:09,034 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:09,451 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312d84cf7cbf434893821c34a432c24f41 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312d84cf7cbf434893821c34a432c24f41 2023-02-09 14:54:09,858 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/bab93e9f2dab4b69837f7a2d75187e04 2023-02-09 14:54:09,865 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/bab93e9f2dab4b69837f7a2d75187e04 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04 2023-02-09 14:54:09,870 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 14:54:09,871 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 837ms, sequenceid=25, compaction requested=true 2023-02-09 14:54:09,871 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:09,871 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:09,871 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:09,871 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:09,871 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,872 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:09,872 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:09,874 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:09,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:09,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:09,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:09,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:09,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58f7fd84[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:09,876 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:09,876 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,877 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,877 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:09,877 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:09,877 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,877 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:09,877 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:09,880 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,880 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:09,880 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:09,880 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:09,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:09,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:09,882 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:09,882 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:09,883 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:09,883 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,883 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:09,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:09,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,886 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:09,887 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:09,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,888 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:09,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:09,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:09,891 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:09,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:09,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:09,891 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:09,892 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:09,891 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,892 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:09,892 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:09,992 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:09,992 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:09,992 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:09,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:09,998 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:09,998 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:09,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:10,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:10,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,002 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:10,002 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,003 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:10,003 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:10,003 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,003 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,005 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:10,005 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:10,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:10,005 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,005 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:10,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:10,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,007 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:10,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:10,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:10,007 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:10,007 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:10,008 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:10,035 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c276a28fa2f14d87aef6bd3d76dd6ffd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c276a28fa2f14d87aef6bd3d76dd6ffd 2023-02-09 14:54:10,044 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/7418c5acf1bb4e618ea53c0d7c0bf783 2023-02-09 14:54:10,050 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/7418c5acf1bb4e618ea53c0d7c0bf783 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783 2023-02-09 14:54:10,064 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 14:54:10,066 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 59ms, sequenceid=29, compaction requested=true 2023-02-09 14:54:10,067 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:10,067 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:10,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:10,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:10,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:10,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:10,070 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:10,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:10,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bfeb3fb[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:10,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:10,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,074 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:10,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:10,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:10,075 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,075 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:10,077 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:10,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:10,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:10,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:10,086 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:10,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:10,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:10,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,086 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:10,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:10,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:10,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,090 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,093 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,097 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:10,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:10,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:10,097 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:10,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:10,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:10,097 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,097 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:10,097 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:10,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,098 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,104 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:10,104 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:10,204 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:10,204 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,205 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:10,210 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,210 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:10,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:10,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,216 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:10,216 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,216 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:10,216 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:10,217 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,217 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,218 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:10,218 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:10,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:10,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,219 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,219 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:10,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:10,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:10,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:10,224 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:10,224 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:10,224 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:10,224 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:10,224 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:10,644 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030c002e68cf3544019b8bdb7ba6962915 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c002e68cf3544019b8bdb7ba6962915 2023-02-09 14:54:10,647 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:10,651 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/8af70c059d1b4e0681db25456dea7bbd 2023-02-09 14:54:10,657 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/8af70c059d1b4e0681db25456dea7bbd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd 2023-02-09 14:54:10,663 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 14:54:10,664 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 440ms, sequenceid=33, compaction requested=true 2023-02-09 14:54:10,664 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:10,664 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:10,664 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:10,664 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:10,664 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,664 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:10,664 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:10,666 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,666 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f9875f4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:10,668 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:10,668 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,669 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,670 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:10,670 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:10,670 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,670 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:10,670 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:10,672 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:10,672 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,672 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,672 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:10,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:10,675 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:10,675 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:10,675 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:10,675 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,675 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:10,677 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,677 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,677 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:10,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:10,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:10,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:10,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:10,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,680 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:10,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:10,682 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:10,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:10,684 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:10,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:10,685 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:10,684 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:10,685 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:10,785 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:10,785 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,785 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:10,789 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,790 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,790 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:10,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:10,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,792 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:10,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:10,792 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:10,793 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,793 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,794 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:10,794 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:10,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:10,795 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:10,795 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:10,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,795 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,796 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,796 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:10,796 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:10,796 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:10,796 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:10,796 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:10,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:10,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:10,797 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:10,797 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:10,797 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:11,224 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090752e01bf235ff4ed6985688648aa6fc26 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090752e01bf235ff4ed6985688648aa6fc26 2023-02-09 14:54:11,229 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/4c12785d7198474780a67b950edc4f2c 2023-02-09 14:54:11,234 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/4c12785d7198474780a67b950edc4f2c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c 2023-02-09 14:54:11,239 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 14:54:11,240 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 443ms, sequenceid=37, compaction requested=true 2023-02-09 14:54:11,240 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:11,240 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:11,240 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:11,240 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:11,240 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,241 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:11,241 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:11,243 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,243 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,243 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,243 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:11,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f9dc12d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:11,245 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:11,245 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,253 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,253 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:11,253 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:11,253 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:11,253 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,253 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:11,254 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:11,254 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,254 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,255 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:11,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:11,257 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:11,257 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:11,257 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:11,257 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,257 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:11,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,260 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,261 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,264 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:11,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:11,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,264 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:11,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,264 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,265 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:11,265 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:11,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,265 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,265 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,265 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,365 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:11,365 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,365 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:11,370 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,370 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:11,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:11,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,372 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:11,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:11,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:11,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,376 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,376 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,376 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:11,376 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:11,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:11,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:11,377 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:11,377 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,377 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:11,378 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:11,378 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:11,378 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:11,378 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. started... 2023-02-09 14:54:11,378 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9a6c61f775706f7023a02aced32af95c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:11,792 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120dd664538b5ca47b59910305443ca5e38 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120dd664538b5ca47b59910305443ca5e38 2023-02-09 14:54:12,199 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/8d9631592f0c4058aa29ce8fae19ba4e 2023-02-09 14:54:12,207 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/8d9631592f0c4058aa29ce8fae19ba4e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e 2023-02-09 14:54:12,213 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 14:54:12,214 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9a6c61f775706f7023a02aced32af95c in 836ms, sequenceid=41, compaction requested=true 2023-02-09 14:54:12,214 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:12,214 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:12,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:12,214 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:12,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,216 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,216 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,216 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:12,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8e10a87[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,218 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:12,218 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,219 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:12,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:12,220 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:12,220 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,220 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,221 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:12,221 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,221 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,222 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:12,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,226 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:12,226 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,226 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,226 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,226 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,229 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,229 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,229 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,229 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,231 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,233 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,233 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,233 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,233 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,233 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,233 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 14:54:12,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,234 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:12,234 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,234 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,234 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,235 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:12,235 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,335 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,335 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,335 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:12,337 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-09 14:54:12,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:12,340 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:12,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:12,341 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955052341 2023-02-09 14:54:12,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:54:12,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 14:54:12,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:12,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x2a81a57d to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:12,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bb6aa1e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:12,348 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,198 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010391d020e7c6164d219e00a98e20540bed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010391d020e7c6164d219e00a98e20540bed 2023-02-09 14:54:13,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:13,212 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:13,214 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32928, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:13,220 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,221 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,221 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:13,223 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59458, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:13,232 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1ed48626bd93401396f9ebacc6eab21b first=Optional[r3] last=Optional[r6] 2023-02-09 14:54:13,232 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1ed48626bd93401396f9ebacc6eab21b}] 2023-02-09 14:54:13,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1ed48626bd93401396f9ebacc6eab21b for inclusion in 9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:13,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 14:54:13,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:13,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:13,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1ed48626bd93401396f9ebacc6eab21b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__el5h4h8hlmcor9qgmoj8ehu2ppkji2633nurjl78602s0ap5ic7b05tgcuoa0u1f/family1/1ed48626bd93401396f9ebacc6eab21b 2023-02-09 14:54:13,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__el5h4h8hlmcor9qgmoj8ehu2ppkji2633nurjl78602s0ap5ic7b05tgcuoa0u1f/family1/1ed48626bd93401396f9ebacc6eab21b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_ 2023-02-09 14:54:13,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__el5h4h8hlmcor9qgmoj8ehu2ppkji2633nurjl78602s0ap5ic7b05tgcuoa0u1f/family1/1ed48626bd93401396f9ebacc6eab21b into 9a6c61f775706f7023a02aced32af95c/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_ - updating store file list. 2023-02-09 14:54:13,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_ into 9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:13,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__el5h4h8hlmcor9qgmoj8ehu2ppkji2633nurjl78602s0ap5ic7b05tgcuoa0u1f/family1/1ed48626bd93401396f9ebacc6eab21b into 9a6c61f775706f7023a02aced32af95c/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_) 2023-02-09 14:54:13,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__el5h4h8hlmcor9qgmoj8ehu2ppkji2633nurjl78602s0ap5ic7b05tgcuoa0u1f/family1/1ed48626bd93401396f9ebacc6eab21b 2023-02-09 14:54:13,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:54:13,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:54:13,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 9a6c61f775706f7023a02aced32af95c/family1 is initiating minor compaction 2023-02-09 14:54:13,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@15932f27; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:13,252 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a6c61f775706f7023a02aced32af95c/family1 in testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:54:13,252 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp, totalSize=50.0 K 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 720e8898e07842869b7ee538ff167059, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting dae99a8a2730467593c52a1a739e9ed9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 58fd78c567c049c69b49573fa9530a36, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 45ec95573fa743f5bafe2112832fe7d5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting bab93e9f2dab4b69837f7a2d75187e04, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 7418c5acf1bb4e618ea53c0d7c0bf783, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8af70c059d1b4e0681db25456dea7bbd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4c12785d7198474780a67b950edc4f2c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8d9631592f0c4058aa29ce8fae19ba4e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:54:13,253 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:54:13,255 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:13,256 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:13,257 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3ecdf3caf644285890137161f6bcee9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228d3ecdf3caf644285890137161f6bcee9 2023-02-09 14:54:13,259 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312d84cf7cbf434893821c34a432c24f41 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512312d84cf7cbf434893821c34a432c24f41 2023-02-09 14:54:13,260 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c276a28fa2f14d87aef6bd3d76dd6ffd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c276a28fa2f14d87aef6bd3d76dd6ffd 2023-02-09 14:54:13,262 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c002e68cf3544019b8bdb7ba6962915 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030c002e68cf3544019b8bdb7ba6962915 2023-02-09 14:54:13,262 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 14:54:13,262 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 14:54:13,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,295 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9a6c61f775706f7023a02aced32af95c#family1#compaction#34 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:13,326 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053f857d46550741bfb5e6bcebb1f772f0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512053f857d46550741bfb5e6bcebb1f772f0 2023-02-09 14:54:13,327 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:13,346 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,346 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:13,353 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9cee6e37f19f4b60a27339c0b70dece3 first=Optional[r1] last=Optional[r2] 2023-02-09 14:54:13,354 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9cee6e37f19f4b60a27339c0b70dece3}] 2023-02-09 14:54:13,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9cee6e37f19f4b60a27339c0b70dece3 for inclusion in 9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:13,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 14:54:13,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:13,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:13,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9cee6e37f19f4b60a27339c0b70dece3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__m9rone38al2fefnvu1t5atk1ut9nhh9gtppg69g0rg4e13mhu79vtkc1v0hna8ns/family1/9cee6e37f19f4b60a27339c0b70dece3 2023-02-09 14:54:13,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__m9rone38al2fefnvu1t5atk1ut9nhh9gtppg69g0rg4e13mhu79vtkc1v0hna8ns/family1/9cee6e37f19f4b60a27339c0b70dece3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/b594e90902444d89aa49a1677891dfd1_SeqId_46_ 2023-02-09 14:54:13,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__m9rone38al2fefnvu1t5atk1ut9nhh9gtppg69g0rg4e13mhu79vtkc1v0hna8ns/family1/9cee6e37f19f4b60a27339c0b70dece3 into 9a6c61f775706f7023a02aced32af95c/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/b594e90902444d89aa49a1677891dfd1_SeqId_46_ - updating store file list. 2023-02-09 14:54:13,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/b594e90902444d89aa49a1677891dfd1_SeqId_46_ into 9a6c61f775706f7023a02aced32af95c/family1 2023-02-09 14:54:13,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__m9rone38al2fefnvu1t5atk1ut9nhh9gtppg69g0rg4e13mhu79vtkc1v0hna8ns/family1/9cee6e37f19f4b60a27339c0b70dece3 into 9a6c61f775706f7023a02aced32af95c/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/b594e90902444d89aa49a1677891dfd1_SeqId_46_) 2023-02-09 14:54:13,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithWeeklyPolicy__m9rone38al2fefnvu1t5atk1ut9nhh9gtppg69g0rg4e13mhu79vtkc1v0hna8ns/family1/9cee6e37f19f4b60a27339c0b70dece3 2023-02-09 14:54:13,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:13,373 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:13,374 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aea112472f364971bc8cf6170d4c43ba to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aea112472f364971bc8cf6170d4c43ba 2023-02-09 14:54:13,375 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120592588665927c4c6a86cf50d352abf4d3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120592588665927c4c6a86cf50d352abf4d3 2023-02-09 14:54:13,376 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-09 14:54:13,376 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 14:54:13,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:13,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:13,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:13,377 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:13,379 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0390 sec 2023-02-09 14:54:13,452 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:13,453 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a81a57d to 127.0.0.1:56347 2023-02-09 14:54:13,453 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:13,461 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:13,461 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06468800 to 127.0.0.1:56347 2023-02-09 14:54:13,461 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:13,461 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:13,461 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2bd50e12 to 127.0.0.1:56347 2023-02-09 14:54:13,461 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:13,461 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:13,462 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6059c336 to 127.0.0.1:56347 2023-02-09 14:54:13,462 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:13,520 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257 (was 260), OpenFileDescriptor=617 (was 612) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=305 (was 291) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3360 (was 3651) 2023-02-09 14:54:13,533 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257, OpenFileDescriptor=617, MaxFileDescriptor=60000, SystemLoadAverage=305, ProcessCount=168, AvailableMemoryMB=3359 2023-02-09 14:54:13,534 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,534 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:13,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 14:54:13,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:13,541 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:13,542 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:13,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-09 14:54:13,543 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,543 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:13,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 14:54:13,545 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:13,545 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:13,546 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:13,546 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6 empty. 2023-02-09 14:54:13,546 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8 empty. 2023-02-09 14:54:13,546 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d empty. 2023-02-09 14:54:13,546 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:13,547 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:13,547 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:13,547 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-09 14:54:13,562 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:13,564 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => b989787f43967a90c0c5150dc576648d, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:13,564 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 55b8d1f0f9f6d2db14ef80187e2488f8, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:13,565 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => c077c04bfcb4d7160a27e1f318e339f6, NAME => 'testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:13,644 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,644 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:13,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 14:54:13,725 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/.tmp/family1/ab372c0f1bf34c559bfad1a49c8383a0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/ab372c0f1bf34c559bfad1a49c8383a0 2023-02-09 14:54:13,731 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9a6c61f775706f7023a02aced32af95c/family1 of 9a6c61f775706f7023a02aced32af95c into ab372c0f1bf34c559bfad1a49c8383a0(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:13,731 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a6c61f775706f7023a02aced32af95c: 2023-02-09 14:54:13,731 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675954453252; duration=0sec 2023-02-09 14:54:13,731 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:13,747 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_] to archive 2023-02-09 14:54:13,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:13,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/720e8898e07842869b7ee538ff167059 2023-02-09 14:54:13,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/dae99a8a2730467593c52a1a739e9ed9 2023-02-09 14:54:13,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/58fd78c567c049c69b49573fa9530a36 2023-02-09 14:54:13,753 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/45ec95573fa743f5bafe2112832fe7d5 2023-02-09 14:54:13,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/bab93e9f2dab4b69837f7a2d75187e04 2023-02-09 14:54:13,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/7418c5acf1bb4e618ea53c0d7c0bf783 2023-02-09 14:54:13,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8af70c059d1b4e0681db25456dea7bbd 2023-02-09 14:54:13,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/4c12785d7198474780a67b950edc4f2c 2023-02-09 14:54:13,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/8d9631592f0c4058aa29ce8fae19ba4e 2023-02-09 14:54:13,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/family1/d8c19f4e2f3d4c1490f9d9658e4fdb16_SeqId_44_ 2023-02-09 14:54:13,846 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:13,846 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:13,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 55b8d1f0f9f6d2db14ef80187e2488f8, disabling compactions & flushes 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing c077c04bfcb4d7160a27e1f318e339f6, disabling compactions & flushes 2023-02-09 14:54:13,977 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:13,977 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. after waiting 0 ms 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:13,977 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. after waiting 0 ms 2023-02-09 14:54:13,977 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing b989787f43967a90c0c5150dc576648d, disabling compactions & flushes 2023-02-09 14:54:13,978 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 55b8d1f0f9f6d2db14ef80187e2488f8: Waiting for close lock at 1675954453977Disabling compacts and flushes for region at 1675954453977Disabling writes for close at 1675954453977Writing region close event to WAL at 1675954453977Closed at 1675954453977 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for c077c04bfcb4d7160a27e1f318e339f6: Waiting for close lock at 1675954453977Disabling compacts and flushes for region at 1675954453977Disabling writes for close at 1675954453978 (+1 ms)Writing region close event to WAL at 1675954453978Closed at 1675954453978 2023-02-09 14:54:13,978 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. after waiting 0 ms 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:13,978 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:13,978 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for b989787f43967a90c0c5150dc576648d: Waiting for close lock at 1675954453978Disabling compacts and flushes for region at 1675954453978Disabling writes for close at 1675954453978Writing region close event to WAL at 1675954453978Closed at 1675954453978 2023-02-09 14:54:13,980 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:13,981 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954453981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954453981"}]},"ts":"1675954453981"} 2023-02-09 14:54:13,982 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954453981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954453981"}]},"ts":"1675954453981"} 2023-02-09 14:54:13,982 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954453981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954453981"}]},"ts":"1675954453981"} 2023-02-09 14:54:13,984 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 14:54:13,985 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:13,985 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954453985"}]},"ts":"1675954453985"} 2023-02-09 14:54:13,987 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-09 14:54:13,990 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c077c04bfcb4d7160a27e1f318e339f6, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=55b8d1f0f9f6d2db14ef80187e2488f8, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b989787f43967a90c0c5150dc576648d, ASSIGN}] 2023-02-09 14:54:13,991 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c077c04bfcb4d7160a27e1f318e339f6, ASSIGN 2023-02-09 14:54:13,991 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=55b8d1f0f9f6d2db14ef80187e2488f8, ASSIGN 2023-02-09 14:54:13,992 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b989787f43967a90c0c5150dc576648d, ASSIGN 2023-02-09 14:54:13,992 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b989787f43967a90c0c5150dc576648d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:13,992 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c077c04bfcb4d7160a27e1f318e339f6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:13,992 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=55b8d1f0f9f6d2db14ef80187e2488f8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:14,144 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c077c04bfcb4d7160a27e1f318e339f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,144 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=b989787f43967a90c0c5150dc576648d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=55b8d1f0f9f6d2db14ef80187e2488f8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,144 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954454144"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954454144"}]},"ts":"1675954454144"} 2023-02-09 14:54:14,144 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954454144"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954454144"}]},"ts":"1675954454144"} 2023-02-09 14:54:14,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954454144"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954454144"}]},"ts":"1675954454144"} 2023-02-09 14:54:14,146 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure c077c04bfcb4d7160a27e1f318e339f6, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:14,147 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:14,147 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:14,147 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure b989787f43967a90c0c5150dc576648d, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:14,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 55b8d1f0f9f6d2db14ef80187e2488f8, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:14,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 14:54:14,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:14,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 55b8d1f0f9f6d2db14ef80187e2488f8, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 14:54:14,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:14,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,303 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,304 DEBUG [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/family1 2023-02-09 14:54:14,305 DEBUG [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/family1 2023-02-09 14:54:14,305 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 55b8d1f0f9f6d2db14ef80187e2488f8 columnFamilyName family1 2023-02-09 14:54:14,306 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] regionserver.HStore(310): Store=55b8d1f0f9f6d2db14ef80187e2488f8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,307 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,308 DEBUG [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/family2 2023-02-09 14:54:14,308 DEBUG [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/family2 2023-02-09 14:54:14,308 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 55b8d1f0f9f6d2db14ef80187e2488f8 columnFamilyName family2 2023-02-09 14:54:14,310 INFO [StoreOpener-55b8d1f0f9f6d2db14ef80187e2488f8-1] regionserver.HStore(310): Store=55b8d1f0f9f6d2db14ef80187e2488f8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:14,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:54:14,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:14,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 55b8d1f0f9f6d2db14ef80187e2488f8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61308115, jitterRate=-0.0864378958940506}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:14,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 55b8d1f0f9f6d2db14ef80187e2488f8: Running coprocessor pre-open hook at 1675954454302Writing region info on filesystem at 1675954454302Initializing all the Stores at 1675954454303 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454303Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454303Cleaning up temporary data from old regions at 1675954454312 (+9 ms)Cleaning up detritus from prior splits at 1675954454312Running coprocessor post-open hooks at 1675954454316 (+4 ms)Region opened successfully at 1675954454316 2023-02-09 14:54:14,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8., pid=33, masterSystemTime=1675954454298 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:14,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:14,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c077c04bfcb4d7160a27e1f318e339f6, NAME => 'testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,319 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=55b8d1f0f9f6d2db14ef80187e2488f8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:14,319 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954454319"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954454319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954454319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954454319"}]},"ts":"1675954454319"} 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,321 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,322 DEBUG [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/family1 2023-02-09 14:54:14,322 DEBUG [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/family1 2023-02-09 14:54:14,323 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-09 14:54:14,323 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 55b8d1f0f9f6d2db14ef80187e2488f8, server=jenkins-hbase4.apache.org,43455,1675954416872 in 173 msec 2023-02-09 14:54:14,323 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c077c04bfcb4d7160a27e1f318e339f6 columnFamilyName family1 2023-02-09 14:54:14,324 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] regionserver.HStore(310): Store=c077c04bfcb4d7160a27e1f318e339f6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,324 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=55b8d1f0f9f6d2db14ef80187e2488f8, ASSIGN in 333 msec 2023-02-09 14:54:14,324 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,326 DEBUG [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/family2 2023-02-09 14:54:14,326 DEBUG [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/family2 2023-02-09 14:54:14,327 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c077c04bfcb4d7160a27e1f318e339f6 columnFamilyName family2 2023-02-09 14:54:14,328 INFO [StoreOpener-c077c04bfcb4d7160a27e1f318e339f6-1] regionserver.HStore(310): Store=c077c04bfcb4d7160a27e1f318e339f6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,331 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:14,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:54:14,334 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:14,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c077c04bfcb4d7160a27e1f318e339f6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66994106, jitterRate=-0.0017100274562835693}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:14,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c077c04bfcb4d7160a27e1f318e339f6: Running coprocessor pre-open hook at 1675954454320Writing region info on filesystem at 1675954454320Initializing all the Stores at 1675954454320Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454320Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454321 (+1 ms)Cleaning up temporary data from old regions at 1675954454330 (+9 ms)Cleaning up detritus from prior splits at 1675954454330Running coprocessor post-open hooks at 1675954454335 (+5 ms)Region opened successfully at 1675954454335 2023-02-09 14:54:14,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6., pid=31, masterSystemTime=1675954454298 2023-02-09 14:54:14,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:14,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:14,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:14,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b989787f43967a90c0c5150dc576648d, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 14:54:14,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,338 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c077c04bfcb4d7160a27e1f318e339f6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:14,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,338 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954454338"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954454338"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954454338"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954454338"}]},"ts":"1675954454338"} 2023-02-09 14:54:14,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,340 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,342 DEBUG [StoreOpener-b989787f43967a90c0c5150dc576648d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1 2023-02-09 14:54:14,342 DEBUG [StoreOpener-b989787f43967a90c0c5150dc576648d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1 2023-02-09 14:54:14,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-09 14:54:14,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure c077c04bfcb4d7160a27e1f318e339f6, server=jenkins-hbase4.apache.org,43455,1675954416872 in 194 msec 2023-02-09 14:54:14,343 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b989787f43967a90c0c5150dc576648d columnFamilyName family1 2023-02-09 14:54:14,344 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c077c04bfcb4d7160a27e1f318e339f6, ASSIGN in 353 msec 2023-02-09 14:54:14,344 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] regionserver.HStore(310): Store=b989787f43967a90c0c5150dc576648d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,345 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,346 DEBUG [StoreOpener-b989787f43967a90c0c5150dc576648d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family2 2023-02-09 14:54:14,346 DEBUG [StoreOpener-b989787f43967a90c0c5150dc576648d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family2 2023-02-09 14:54:14,347 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b989787f43967a90c0c5150dc576648d columnFamilyName family2 2023-02-09 14:54:14,347 INFO [StoreOpener-b989787f43967a90c0c5150dc576648d-1] regionserver.HStore(310): Store=b989787f43967a90c0c5150dc576648d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:14,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:14,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b989787f43967a90c0c5150dc576648d 2023-02-09 14:54:14,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:14,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b989787f43967a90c0c5150dc576648d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69981927, jitterRate=0.04281197488307953}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b989787f43967a90c0c5150dc576648d: Running coprocessor pre-open hook at 1675954454339Writing region info on filesystem at 1675954454339Initializing all the Stores at 1675954454339Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454340 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954454340Cleaning up temporary data from old regions at 1675954454349 (+9 ms)Cleaning up detritus from prior splits at 1675954454350 (+1 ms)Running coprocessor post-open hooks at 1675954454354 (+4 ms)Region opened successfully at 1675954454354 2023-02-09 14:54:14,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d., pid=32, masterSystemTime=1675954454298 2023-02-09 14:54:14,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:14,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:14,356 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=b989787f43967a90c0c5150dc576648d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,357 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954454356"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954454356"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954454356"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954454356"}]},"ts":"1675954454356"} 2023-02-09 14:54:14,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-02-09 14:54:14,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure b989787f43967a90c0c5150dc576648d, server=jenkins-hbase4.apache.org,43455,1675954416872 in 211 msec 2023-02-09 14:54:14,361 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-02-09 14:54:14,361 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b989787f43967a90c0c5150dc576648d, ASSIGN in 370 msec 2023-02-09 14:54:14,362 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:14,362 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954454362"}]},"ts":"1675954454362"} 2023-02-09 14:54:14,363 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-09 14:54:14,366 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:14,368 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 829 msec 2023-02-09 14:54:14,396 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-09 14:54:14,648 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:14,648 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:14,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 14:54:14,649 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-09 14:54:14,655 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:14,655 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:14,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:14,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,658 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 14:54:14,658 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:14,658 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 14:54:14,658 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 14:54:14,658 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,658 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,660 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,660 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:14,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:14,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:14,660 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:14,660 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:14,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:14,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,661 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:14,661 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 14:54:14,661 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:14,661 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 14:54:14,661 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:14,661 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. started... 2023-02-09 14:54:14,661 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:54:14,661 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:14,661 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:14,662 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. started... 2023-02-09 14:54:14,661 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:14,662 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 55b8d1f0f9f6d2db14ef80187e2488f8: 2023-02-09 14:54:14,663 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:14,662 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. started... 2023-02-09 14:54:14,663 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:54:14,663 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b989787f43967a90c0c5150dc576648d 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-09 14:54:14,663 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for c077c04bfcb4d7160a27e1f318e339f6: 2023-02-09 14:54:14,663 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:14,663 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:54:15,076 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020912ee5ac1ea7044cb864d0784dda8f8bf to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020912ee5ac1ea7044cb864d0784dda8f8bf 2023-02-09 14:54:15,482 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp/family1/1a17c2fc77874f3aa868d4931b746be3 2023-02-09 14:54:15,491 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp/family1/1a17c2fc77874f3aa868d4931b746be3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3 2023-02-09 14:54:15,497 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 14:54:15,498 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for b989787f43967a90c0c5150dc576648d in 835ms, sequenceid=5, compaction requested=false 2023-02-09 14:54:15,499 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b989787f43967a90c0c5150dc576648d: 2023-02-09 14:54:15,499 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 14:54:15,501 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,501 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:15,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:15,501 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,501 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 14:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66ee010b[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:15,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 14:54:15,503 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,504 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 14:54:15,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 14:54:15,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:15,505 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,505 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:15,508 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 14:54:15,508 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:15,508 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 14:54:15,508 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:15,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 14:54:15,515 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 14:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:15,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,515 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:15,520 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,520 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:15,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,520 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:15,521 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:15,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,532 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,533 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:15,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 14:54:15,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 14:54:15,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:15,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:15,534 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:15,533 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:15,534 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:15,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:15,634 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:15,634 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:15,634 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:15,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:15,641 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:15,641 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:15,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:15,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:15,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,644 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 14:54:15,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:15,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 14:54:15,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 14:54:15,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,647 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:15,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:15,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:15,647 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:15,647 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:15,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-09 14:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:15,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 14:54:15,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:15,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. started... 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:15,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:15,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 55b8d1f0f9f6d2db14ef80187e2488f8: 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. started... 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:54:15,649 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. started... 2023-02-09 14:54:15,650 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:54:15,650 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for c077c04bfcb4d7160a27e1f318e339f6: 2023-02-09 14:54:15,650 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b989787f43967a90c0c5150dc576648d 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-09 14:54:15,650 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:54:15,650 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:54:15,668 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020916a0369a65ae4ce89ed49667af39a175 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020916a0369a65ae4ce89ed49667af39a175 2023-02-09 14:54:16,088 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp/family1/aacc3cd80a5f448caf24c8d679a2f9e2 2023-02-09 14:54:16,093 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp/family1/aacc3cd80a5f448caf24c8d679a2f9e2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2 2023-02-09 14:54:16,097 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2, entries=2, sequenceid=9, filesize=5.0 K 2023-02-09 14:54:16,098 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for b989787f43967a90c0c5150dc576648d in 448ms, sequenceid=9, compaction requested=false 2023-02-09 14:54:16,098 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b989787f43967a90c0c5150dc576648d: 2023-02-09 14:54:16,099 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:16,099 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:54:16,099 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:54:16,099 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,099 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 14:54:16,099 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 14:54:16,100 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,100 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:16,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:16,101 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,101 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 14:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ab5eff7[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:16,102 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 14:54:16,102 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,104 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 14:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 14:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:16,104 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,104 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:16,105 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 14:54:16,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:16,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:16,106 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 14:54:16,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:16,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:16,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:16,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:16,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 14:54:16,108 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 14:54:16,108 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:16,108 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:16,108 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,108 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:16,111 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,111 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:16,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:16,111 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:16,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:16,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:16,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:16,112 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:16,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,113 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:16,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:16,115 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:16,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 14:54:16,116 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:16,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 14:54:16,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:16,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:16,116 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:16,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:16,117 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,117 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,117 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,117 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:16,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:16,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:16,117 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:16,217 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:16,217 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,217 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:16,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:16,225 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-09 14:54:16,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 14:54:16,227 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 14:54:16,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:16,228 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955056228 2023-02-09 14:54:16,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:16,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-09 14:54:16,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 14:54:16,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x5f954bdf to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:16,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@715dfc69, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:16,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 14:54:16,683 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302094b7f910acb0f447ca021b0db3465bdb3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302094b7f910acb0f447ca021b0db3465bdb3 2023-02-09 14:54:16,684 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 14:54:16,700 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:16,701 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53212, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:16,717 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,717 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,717 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:16,720 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55404, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:16,752 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/47003a0d268b48cabb84d7172ddb125a first=Optional[k0] last=Optional[k1] 2023-02-09 14:54:16,753 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/47003a0d268b48cabb84d7172ddb125a}] 2023-02-09 14:54:16,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/47003a0d268b48cabb84d7172ddb125a for inclusion in b989787f43967a90c0c5150dc576648d/family1 2023-02-09 14:54:16,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-09 14:54:16,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 14:54:16,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for b989787f43967a90c0c5150dc576648d: 2023-02-09 14:54:16,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/47003a0d268b48cabb84d7172ddb125a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testScannerOnBulkLoadRefHFiles__ojm4hura0vin82hdoo4tsv4vh21rhna0l95iumm5qd8u1pfasbbmjjggh92o7sl7/family1/47003a0d268b48cabb84d7172ddb125a 2023-02-09 14:54:16,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testScannerOnBulkLoadRefHFiles__ojm4hura0vin82hdoo4tsv4vh21rhna0l95iumm5qd8u1pfasbbmjjggh92o7sl7/family1/47003a0d268b48cabb84d7172ddb125a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_ 2023-02-09 14:54:16,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testScannerOnBulkLoadRefHFiles__ojm4hura0vin82hdoo4tsv4vh21rhna0l95iumm5qd8u1pfasbbmjjggh92o7sl7/family1/47003a0d268b48cabb84d7172ddb125a into b989787f43967a90c0c5150dc576648d/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_ - updating store file list. 2023-02-09 14:54:16,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_ into b989787f43967a90c0c5150dc576648d/family1 2023-02-09 14:54:16,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testScannerOnBulkLoadRefHFiles__ojm4hura0vin82hdoo4tsv4vh21rhna0l95iumm5qd8u1pfasbbmjjggh92o7sl7/family1/47003a0d268b48cabb84d7172ddb125a into b989787f43967a90c0c5150dc576648d/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_) 2023-02-09 14:54:16,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testScannerOnBulkLoadRefHFiles__ojm4hura0vin82hdoo4tsv4vh21rhna0l95iumm5qd8u1pfasbbmjjggh92o7sl7/family1/47003a0d268b48cabb84d7172ddb125a 2023-02-09 14:54:16,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:54:16,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:54:16,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): b989787f43967a90c0c5150dc576648d/family1 is initiating minor compaction (all files) 2023-02-09 14:54:16,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@710a2178; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,778 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b989787f43967a90c0c5150dc576648d/family1 in testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:54:16,778 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp, totalSize=15.1 K 2023-02-09 14:54:16,789 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 1a17c2fc77874f3aa868d4931b746be3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954454650 2023-02-09 14:54:16,789 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 14:54:16,789 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting aacc3cd80a5f448caf24c8d679a2f9e2, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954454650 2023-02-09 14:54:16,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:16,790 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting d851c2b61cb44602b1ef81aca6af7216_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954454650 2023-02-09 14:54:16,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020912ee5ac1ea7044cb864d0784dda8f8bf to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020912ee5ac1ea7044cb864d0784dda8f8bf 2023-02-09 14:54:16,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020916a0369a65ae4ce89ed49667af39a175 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020916a0369a65ae4ce89ed49667af39a175 2023-02-09 14:54:16,793 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-09 14:54:16,793 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 14:54:16,794 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 14:54:16,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 14:54:16,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:16,798 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 14:54:16,800 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 572 msec 2023-02-09 14:54:16,805 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): b989787f43967a90c0c5150dc576648d#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:16,834 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257 (was 257), OpenFileDescriptor=603 (was 617), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 305), ProcessCount=168 (was 168), AvailableMemoryMB=3246 (was 3359) 2023-02-09 14:54:16,850 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257, OpenFileDescriptor=603, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=168, AvailableMemoryMB=3246 2023-02-09 14:54:16,851 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,852 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:16,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:54:16,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:16,856 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:16,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-09 14:54:16,857 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,857 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:16,857 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:16,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:16,860 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:16,860 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed empty. 2023-02-09 14:54:16,861 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:16,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-09 14:54:16,958 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:16,958 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:16,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,159 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,159 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,275 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:17,278 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 773391e6c583b5ff2a5d0746351f3fed, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 773391e6c583b5ff2a5d0746351f3fed, disabling compactions & flushes 2023-02-09 14:54:17,285 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. after waiting 0 ms 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,285 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,285 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 773391e6c583b5ff2a5d0746351f3fed: Waiting for close lock at 1675954457285Disabling compacts and flushes for region at 1675954457285Disabling writes for close at 1675954457285Writing region close event to WAL at 1675954457285Closed at 1675954457285 2023-02-09 14:54:17,288 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:17,289 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954457289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954457289"}]},"ts":"1675954457289"} 2023-02-09 14:54:17,290 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:17,291 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:17,291 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457291"}]},"ts":"1675954457291"} 2023-02-09 14:54:17,292 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 14:54:17,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=773391e6c583b5ff2a5d0746351f3fed, ASSIGN}] 2023-02-09 14:54:17,298 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=773391e6c583b5ff2a5d0746351f3fed, ASSIGN 2023-02-09 14:54:17,298 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=773391e6c583b5ff2a5d0746351f3fed, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:17,450 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=773391e6c583b5ff2a5d0746351f3fed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:17,450 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954457450"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954457450"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954457450"}]},"ts":"1675954457450"} 2023-02-09 14:54:17,452 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 773391e6c583b5ff2a5d0746351f3fed, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:17,461 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,461 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 773391e6c583b5ff2a5d0746351f3fed, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,607 INFO [StoreOpener-773391e6c583b5ff2a5d0746351f3fed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,608 DEBUG [StoreOpener-773391e6c583b5ff2a5d0746351f3fed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:17,608 DEBUG [StoreOpener-773391e6c583b5ff2a5d0746351f3fed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:17,609 INFO [StoreOpener-773391e6c583b5ff2a5d0746351f3fed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 773391e6c583b5ff2a5d0746351f3fed columnFamilyName family1 2023-02-09 14:54:17,610 INFO [StoreOpener-773391e6c583b5ff2a5d0746351f3fed-1] regionserver.HStore(310): Store=773391e6c583b5ff2a5d0746351f3fed/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:17,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:54:17,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:17,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 773391e6c583b5ff2a5d0746351f3fed; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66511519, jitterRate=-0.008901134133338928}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:17,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 773391e6c583b5ff2a5d0746351f3fed: Running coprocessor pre-open hook at 1675954457606Writing region info on filesystem at 1675954457606Initializing all the Stores at 1675954457607 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954457607Cleaning up temporary data from old regions at 1675954457612 (+5 ms)Cleaning up detritus from prior splits at 1675954457612Running coprocessor post-open hooks at 1675954457617 (+5 ms)Region opened successfully at 1675954457617 2023-02-09 14:54:17,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed., pid=37, masterSystemTime=1675954457603 2023-02-09 14:54:17,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,620 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=773391e6c583b5ff2a5d0746351f3fed, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:17,620 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954457620"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954457620"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954457620"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954457620"}]},"ts":"1675954457620"} 2023-02-09 14:54:17,623 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-09 14:54:17,623 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 773391e6c583b5ff2a5d0746351f3fed, server=jenkins-hbase4.apache.org,43455,1675954416872 in 169 msec 2023-02-09 14:54:17,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-09 14:54:17,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=773391e6c583b5ff2a5d0746351f3fed, ASSIGN in 327 msec 2023-02-09 14:54:17,625 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:17,626 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457625"}]},"ts":"1675954457625"} 2023-02-09 14:54:17,627 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 14:54:17,629 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:17,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 775 msec 2023-02-09 14:54:17,962 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,962 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,963 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-09 14:54:17,985 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,985 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:17,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,988 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,991 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:17,991 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:17,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:17,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:17,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,991 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:17,991 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:17,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:17,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:17,992 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:17,993 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:17,993 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:17,993 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:17,993 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:17,993 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:17,993 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:17,994 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:54:18,012 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bf2360ae48ed44788a60e6c64d5ca8ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bf2360ae48ed44788a60e6c64d5ca8ed 2023-02-09 14:54:18,024 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/.tmp/family1/18ac4f2c26534c7893a2a4a6570d3044 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/18ac4f2c26534c7893a2a4a6570d3044 2023-02-09 14:54:18,030 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b989787f43967a90c0c5150dc576648d/family1 of b989787f43967a90c0c5150dc576648d into 18ac4f2c26534c7893a2a4a6570d3044(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:54:18,030 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b989787f43967a90c0c5150dc576648d: 2023-02-09 14:54:18,030 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1675954456778; duration=1sec 2023-02-09 14:54:18,030 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:18,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_] to archive 2023-02-09 14:54:18,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:18,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/1a17c2fc77874f3aa868d4931b746be3 2023-02-09 14:54:18,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/aacc3cd80a5f448caf24c8d679a2f9e2 2023-02-09 14:54:18,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/family1/d851c2b61cb44602b1ef81aca6af7216_SeqId_12_ 2023-02-09 14:54:18,416 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/925513fc09d94500aebd4dc48ad0869c 2023-02-09 14:54:18,422 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/925513fc09d94500aebd4dc48ad0869c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c 2023-02-09 14:54:18,427 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 14:54:18,428 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 434ms, sequenceid=5, compaction requested=false 2023-02-09 14:54:18,428 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:18,429 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:18,429 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,430 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:18,430 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:18,431 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,431 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:18,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ba0d325[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:18,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,435 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,435 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:18,435 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:18,435 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:18,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,436 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,437 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:18,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:18,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,439 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:18,439 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,439 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,439 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,439 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,442 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,442 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,442 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,443 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,444 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:18,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,446 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:18,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,446 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,547 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,547 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,547 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:18,551 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,552 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:18,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,554 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:18,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:18,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:18,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,555 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,556 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,556 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:18,556 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:18,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:18,557 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:18,557 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,557 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:18,557 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,557 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,557 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:18,557 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:18,557 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:54:18,971 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112810d671b4d62c4dfc944ca897a955dbb1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112810d671b4d62c4dfc944ca897a955dbb1 2023-02-09 14:54:19,377 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/d5cf126b2a87465f9fb7fb89ce3fe41c 2023-02-09 14:54:19,382 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/d5cf126b2a87465f9fb7fb89ce3fe41c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c 2023-02-09 14:54:19,386 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:54:19,387 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 830ms, sequenceid=9, compaction requested=false 2023-02-09 14:54:19,387 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:19,387 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:19,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:19,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:19,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,388 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:19,388 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:19,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,389 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:19,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:19,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b1545e4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:19,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:19,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,395 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:19,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:19,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:19,395 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,395 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:19,396 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:19,396 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,396 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,396 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:19,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:19,398 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:19,398 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:19,398 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:19,398 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,398 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:19,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,400 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,401 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,404 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:19,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:19,404 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:19,405 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:19,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,505 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:19,505 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:19,506 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:19,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:19,510 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:19,510 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:19,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:19,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:19,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,512 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:19,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:19,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:19,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:19,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,514 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:19,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:19,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:19,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:19,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:19,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:19,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:19,516 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:19,516 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:19,516 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:19,516 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:19,803 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:19,929 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201d4d53afdbecb4271969b9199cfb748e4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d4d53afdbecb4271969b9199cfb748e4 2023-02-09 14:54:20,334 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/3017068a77544c30a28a0c887fee9b5c 2023-02-09 14:54:20,339 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/3017068a77544c30a28a0c887fee9b5c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c 2023-02-09 14:54:20,344 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 14:54:20,344 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 828ms, sequenceid=13, compaction requested=true 2023-02-09 14:54:20,345 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:20,345 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:20,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:20,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:20,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:20,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:20,346 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,346 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,346 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:20,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:20,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1de5ebd4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:20,348 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:20,348 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,349 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:20,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:20,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:20,349 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,349 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:20,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:20,352 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:20,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:20,354 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:20,354 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:20,354 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:20,354 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,354 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:20,356 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,356 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,356 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,356 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,357 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:20,360 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:20,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:20,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:20,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:20,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:20,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,460 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:20,461 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:20,461 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:20,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:20,465 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:20,465 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:20,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:20,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:20,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,467 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:20,467 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:20,467 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:20,467 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:20,467 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,468 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,469 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:20,469 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:20,469 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,469 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:20,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,470 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,470 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:20,470 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:20,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:20,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:20,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:20,470 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:20,471 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:20,471 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:20,471 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:20,489 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120557e588060c684d98b659cdc2b844a720 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120557e588060c684d98b659cdc2b844a720 2023-02-09 14:54:20,895 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/79fa5a5c23be43b2974b200a61d876a5 2023-02-09 14:54:20,899 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/79fa5a5c23be43b2974b200a61d876a5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5 2023-02-09 14:54:20,904 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 14:54:20,904 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 433ms, sequenceid=17, compaction requested=true 2023-02-09 14:54:20,904 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:20,905 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:20,905 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:20,905 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:20,905 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,905 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:20,905 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:20,907 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,907 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,908 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,908 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:20,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:20,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@388a65c8[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:20,909 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:20,909 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,910 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,910 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:20,910 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:20,910 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:20,910 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,910 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:20,911 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,911 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:20,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,912 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:20,912 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:20,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:20,914 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:20,914 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:20,914 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:20,914 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,914 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:20,915 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,915 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:20,915 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:20,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:20,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:20,915 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:20,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,917 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:20,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,920 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:20,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:20,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:20,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:20,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,920 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:20,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:20,924 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:20,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:20,924 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:21,024 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:21,025 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:21,025 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:21,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:21,029 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:21,029 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:21,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:21,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:21,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,031 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:21,031 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:21,031 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:21,031 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:21,032 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,032 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,033 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:21,033 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:21,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:21,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,034 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,034 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:21,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:21,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,035 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:21,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,035 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:21,035 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:21,035 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:21,035 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:21,035 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:21,035 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:21,035 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:21,035 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:21,452 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122843f6198cc42b4a41a64baac80286f65f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122843f6198cc42b4a41a64baac80286f65f 2023-02-09 14:54:21,864 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/8ecef094ad224673a2a09f5daa815132 2023-02-09 14:54:21,869 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/8ecef094ad224673a2a09f5daa815132 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132 2023-02-09 14:54:21,873 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 14:54:21,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 839ms, sequenceid=21, compaction requested=true 2023-02-09 14:54:21,874 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:21,874 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:21,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:21,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:21,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:21,874 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:21,883 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@353ea007[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:21,884 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:21,884 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,885 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,885 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:21,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:21,886 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:21,886 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,886 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:21,887 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,887 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:21,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:21,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:21,887 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:21,887 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:21,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:21,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:21,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:21,889 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:21,889 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:21,889 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:21,889 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,889 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:21,890 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,890 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:21,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,890 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:21,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:21,891 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,892 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:21,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:21,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:21,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:21,897 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:21,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:21,897 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:21,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:21,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:21,997 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:21,998 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:21,998 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:21,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:22,002 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:22,002 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:22,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:22,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:22,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,004 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:22,004 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:22,004 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:22,004 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:22,005 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,005 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,006 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:22,006 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:22,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:22,006 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,006 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,007 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:22,008 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:22,008 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:22,008 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:22,421 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231407f07be449d49abb111a569e44ed877 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231407f07be449d49abb111a569e44ed877 2023-02-09 14:54:22,827 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/d09769e62aad459d841ab2fee1ce4d61 2023-02-09 14:54:22,832 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/d09769e62aad459d841ab2fee1ce4d61 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61 2023-02-09 14:54:22,837 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 14:54:22,838 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 830ms, sequenceid=25, compaction requested=true 2023-02-09 14:54:22,838 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:22,838 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:22,840 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:22,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,840 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:22,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:22,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1425e921[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:22,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:22,842 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,843 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:22,843 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,843 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:22,845 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:22,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:22,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:22,848 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:22,848 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:22,848 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:22,848 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,848 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:22,849 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,849 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:22,849 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:22,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:22,850 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:22,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,851 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:22,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:22,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:22,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:22,854 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:22,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,854 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:22,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:22,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:22,855 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:22,955 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:22,955 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:22,955 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:22,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:22,960 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:22,960 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:22,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:22,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:22,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,962 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:22,962 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:22,962 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:22,962 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:22,962 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,964 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,964 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:22,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:22,964 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:22,964 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,965 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,966 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:22,966 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:22,966 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:22,966 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:22,966 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:22,966 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:22,966 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:22,966 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:22,966 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:23,380 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010b340f981e0242a4a7e437c2ebbe309a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601010b340f981e0242a4a7e437c2ebbe309a 2023-02-09 14:54:23,787 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/066de1ac4e214d12843e504f280b62cc 2023-02-09 14:54:23,792 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/066de1ac4e214d12843e504f280b62cc as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc 2023-02-09 14:54:23,795 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 14:54:23,796 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 830ms, sequenceid=29, compaction requested=true 2023-02-09 14:54:23,796 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:23,796 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:23,796 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:23,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:23,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:23,797 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:23,804 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,804 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:23,804 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,804 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@262c0b3a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:23,805 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:23,806 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,807 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:23,807 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,807 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:23,808 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:23,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:23,810 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:23,810 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:23,810 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:23,810 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,810 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:23,812 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,812 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:23,812 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:23,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:23,812 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:23,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,813 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:23,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:23,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:23,822 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:23,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:23,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:23,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:23,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:23,823 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:23,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:23,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:23,923 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:23,923 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:23,923 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:23,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:23,928 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:23,928 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:23,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:23,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:23,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,930 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:23,930 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:23,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:23,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:23,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,932 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:23,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:23,932 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,933 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:23,933 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:23,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:23,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:23,934 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:23,934 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:23,934 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:23,934 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:23,947 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103eda41004c39f47328e43eff6aa0cade0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103eda41004c39f47328e43eff6aa0cade0 2023-02-09 14:54:24,352 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/5d8764c9fed143e3ba81582e91b2ea3d 2023-02-09 14:54:24,357 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/5d8764c9fed143e3ba81582e91b2ea3d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d 2023-02-09 14:54:24,361 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 14:54:24,362 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 428ms, sequenceid=33, compaction requested=true 2023-02-09 14:54:24,362 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:24,362 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:24,362 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:24,363 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,363 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:24,364 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,364 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4de38e12[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:24,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:24,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,366 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:24,366 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:24,366 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:24,366 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,366 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:24,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,370 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:24,370 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:24,370 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:24,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:24,372 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:24,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:24,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:24,372 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,372 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:24,373 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,373 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:24,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,373 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:24,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,375 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:24,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,376 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,376 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:24,376 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,377 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:24,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 14:54:24,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:24,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:24,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:24,377 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:24,477 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:24,477 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:24,478 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:24,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:24,482 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:24,482 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:24,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:24,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:24,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,485 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:24,485 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:24,485 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:24,485 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:24,485 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,485 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,486 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,486 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:24,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:24,487 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:24,487 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:24,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:24,488 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:24,488 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:24,488 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:24,489 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:24,489 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:24,489 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:24,490 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:24,490 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:24,902 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090743777c0738c242338737673e7718a321 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090743777c0738c242338737673e7718a321 2023-02-09 14:54:25,308 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/24ae14aaf8554b6bbc5c9dc0005c0aac 2023-02-09 14:54:25,313 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/24ae14aaf8554b6bbc5c9dc0005c0aac as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac 2023-02-09 14:54:25,317 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 14:54:25,318 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 828ms, sequenceid=37, compaction requested=true 2023-02-09 14:54:25,318 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:25,318 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:25,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:25,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:25,321 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,321 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:25,321 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,321 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d9d2131[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:25,322 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:25,322 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,323 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:25,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:25,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:25,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:25,324 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,324 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:25,327 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:25,327 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:25,327 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:25,327 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:25,327 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,327 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:25,328 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,328 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,328 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:25,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:25,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:25,328 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,329 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:25,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:25,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:25,332 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:25,332 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:25,333 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,333 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,333 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:25,333 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:25,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:25,433 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:25,433 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,433 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:25,438 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,438 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:25,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:25,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,440 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:25,440 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:25,440 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 14:54:25,440 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 14:54:25,440 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,440 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,441 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,441 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:25,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:25,442 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:25,442 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,443 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 14:54:25,443 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:25,443 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 14:54:25,443 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:25,443 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 14:54:25,443 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:25,443 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:25,443 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:25,443 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. started... 2023-02-09 14:54:25,443 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 773391e6c583b5ff2a5d0746351f3fed 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:25,856 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511207954ee27a2de4478aaf36265a0be65da to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511207954ee27a2de4478aaf36265a0be65da 2023-02-09 14:54:26,261 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/68a91b49c1b2474ca7f26de8d1510803 2023-02-09 14:54:26,267 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/68a91b49c1b2474ca7f26de8d1510803 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803 2023-02-09 14:54:26,271 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 14:54:26,271 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 773391e6c583b5ff2a5d0746351f3fed in 828ms, sequenceid=41, compaction requested=true 2023-02-09 14:54:26,272 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:26,272 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:26,272 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:26,272 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:26,272 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,272 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 14:54:26,272 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 14:54:26,274 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,274 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:26,274 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,274 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:26,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:26,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 14:54:26,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@364aad42[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:26,275 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 14:54:26,275 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,276 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 14:54:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 14:54:26,276 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:26,276 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,276 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:26,278 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,278 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 14:54:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:26,278 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:26,279 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 14:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:26,281 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:54:26,281 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 14:54:26,281 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:26,281 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:26,281 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,281 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:26,282 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,282 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:26,282 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:26,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:26,283 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:26,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,284 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:26,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 14:54:26,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:26,289 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:26,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:26,289 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,389 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:26,390 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:26,390 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:26,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:26,392 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-09 14:54:26,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:26,394 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:26,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:26,396 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955066396 2023-02-09 14:54:26,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:26,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 14:54:26,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:26,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x2b501a7a to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:26,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@357015cd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:26,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,255 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037ead2fea1e3b49cf97a3ddb19fad37c2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601037ead2fea1e3b49cf97a3ddb19fad37c2 2023-02-09 14:54:27,256 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:27,271 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:27,272 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59722, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:27,279 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,279 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,280 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:27,281 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:27,289 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/95ff5fa4ceed4f7e992683c6e1d97204 first=Optional[r3] last=Optional[r6] 2023-02-09 14:54:27,290 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/95ff5fa4ceed4f7e992683c6e1d97204}] 2023-02-09 14:54:27,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/95ff5fa4ceed4f7e992683c6e1d97204 for inclusion in 773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:27,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 14:54:27,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:27,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:27,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/95ff5fa4ceed4f7e992683c6e1d97204 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__puu85f2l0su61c8gcnf8jtlge16bv6nku0t5js4e6ev37loe6teuub7sc28lgs3t/family1/95ff5fa4ceed4f7e992683c6e1d97204 2023-02-09 14:54:27,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__puu85f2l0su61c8gcnf8jtlge16bv6nku0t5js4e6ev37loe6teuub7sc28lgs3t/family1/95ff5fa4ceed4f7e992683c6e1d97204 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_ 2023-02-09 14:54:27,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__puu85f2l0su61c8gcnf8jtlge16bv6nku0t5js4e6ev37loe6teuub7sc28lgs3t/family1/95ff5fa4ceed4f7e992683c6e1d97204 into 773391e6c583b5ff2a5d0746351f3fed/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_ - updating store file list. 2023-02-09 14:54:27,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_ into 773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:27,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__puu85f2l0su61c8gcnf8jtlge16bv6nku0t5js4e6ev37loe6teuub7sc28lgs3t/family1/95ff5fa4ceed4f7e992683c6e1d97204 into 773391e6c583b5ff2a5d0746351f3fed/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_) 2023-02-09 14:54:27,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__puu85f2l0su61c8gcnf8jtlge16bv6nku0t5js4e6ev37loe6teuub7sc28lgs3t/family1/95ff5fa4ceed4f7e992683c6e1d97204 2023-02-09 14:54:27,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:54:27,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:54:27,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 773391e6c583b5ff2a5d0746351f3fed/family1 is initiating minor compaction 2023-02-09 14:54:27,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@207c44a4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:27,308 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 773391e6c583b5ff2a5d0746351f3fed/family1 in testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:27,308 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp, totalSize=50.0 K 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting d5cf126b2a87465f9fb7fb89ce3fe41c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 3017068a77544c30a28a0c887fee9b5c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 79fa5a5c23be43b2974b200a61d876a5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8ecef094ad224673a2a09f5daa815132, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting d09769e62aad459d841ab2fee1ce4d61, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 066de1ac4e214d12843e504f280b62cc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 5d8764c9fed143e3ba81582e91b2ea3d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 24ae14aaf8554b6bbc5c9dc0005c0aac, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 68a91b49c1b2474ca7f26de8d1510803, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:54:27,308 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:54:27,310 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:27,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:27,312 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122843f6198cc42b4a41a64baac80286f65f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122843f6198cc42b4a41a64baac80286f65f 2023-02-09 14:54:27,320 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231407f07be449d49abb111a569e44ed877 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231407f07be449d49abb111a569e44ed877 2023-02-09 14:54:27,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601010b340f981e0242a4a7e437c2ebbe309a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601010b340f981e0242a4a7e437c2ebbe309a 2023-02-09 14:54:27,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103eda41004c39f47328e43eff6aa0cade0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103eda41004c39f47328e43eff6aa0cade0 2023-02-09 14:54:27,323 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 14:54:27,323 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:27,345 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 773391e6c583b5ff2a5d0746351f3fed#family1#compaction#48 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:28,163 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/3bb729c2f6ad4aae81f20d26978eeda1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3bb729c2f6ad4aae81f20d26978eeda1 2023-02-09 14:54:28,168 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 773391e6c583b5ff2a5d0746351f3fed/family1 of 773391e6c583b5ff2a5d0746351f3fed into 3bb729c2f6ad4aae81f20d26978eeda1(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:28,168 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:28,169 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675954467308; duration=0sec 2023-02-09 14:54:28,169 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:28,173 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055a3a85b3a6724286ac536ca705f7f56c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055a3a85b3a6724286ac536ca705f7f56c 2023-02-09 14:54:28,175 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:28,192 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,192 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,202 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be07a0dfee4049529b8e6eaa71190391 first=Optional[r00] last=Optional[r2] 2023-02-09 14:54:28,202 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be07a0dfee4049529b8e6eaa71190391}] 2023-02-09 14:54:28,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be07a0dfee4049529b8e6eaa71190391 for inclusion in 773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 14:54:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:28,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/be07a0dfee4049529b8e6eaa71190391 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5qiv5u3ulobeikr5nj799bddlquo6nbpiu90dju7prifejppmi23l1c5u4689ko6/family1/be07a0dfee4049529b8e6eaa71190391 2023-02-09 14:54:28,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5qiv5u3ulobeikr5nj799bddlquo6nbpiu90dju7prifejppmi23l1c5u4689ko6/family1/be07a0dfee4049529b8e6eaa71190391 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_ 2023-02-09 14:54:28,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5qiv5u3ulobeikr5nj799bddlquo6nbpiu90dju7prifejppmi23l1c5u4689ko6/family1/be07a0dfee4049529b8e6eaa71190391 into 773391e6c583b5ff2a5d0746351f3fed/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_ - updating store file list. 2023-02-09 14:54:28,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_ into 773391e6c583b5ff2a5d0746351f3fed/family1 2023-02-09 14:54:28,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5qiv5u3ulobeikr5nj799bddlquo6nbpiu90dju7prifejppmi23l1c5u4689ko6/family1/be07a0dfee4049529b8e6eaa71190391 into 773391e6c583b5ff2a5d0746351f3fed/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_) 2023-02-09 14:54:28,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5qiv5u3ulobeikr5nj799bddlquo6nbpiu90dju7prifejppmi23l1c5u4689ko6/family1/be07a0dfee4049529b8e6eaa71190391 2023-02-09 14:54:28,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:54:28,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:54:28,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 773391e6c583b5ff2a5d0746351f3fed/family1 is initiating minor compaction (all files) 2023-02-09 14:54:28,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@715611ec; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:28,219 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 773391e6c583b5ff2a5d0746351f3fed/family1 in testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:54:28,219 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3bb729c2f6ad4aae81f20d26978eeda1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp, totalSize=337.0 K 2023-02-09 14:54:28,219 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 925513fc09d94500aebd4dc48ad0869c, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:54:28,220 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 3bb729c2f6ad4aae81f20d26978eeda1, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:54:28,220 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840463 2023-02-09 14:54:28,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:28,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:28,223 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bf2360ae48ed44788a60e6c64d5ca8ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bf2360ae48ed44788a60e6c64d5ca8ed 2023-02-09 14:54:28,225 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d4d53afdbecb4271969b9199cfb748e4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201d4d53afdbecb4271969b9199cfb748e4 2023-02-09 14:54:28,227 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120557e588060c684d98b659cdc2b844a720 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120557e588060c684d98b659cdc2b844a720 2023-02-09 14:54:28,227 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-09 14:54:28,227 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 14:54:28,227 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 14:54:28,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 14:54:28,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:28,229 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:54:28,231 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8370 sec 2023-02-09 14:54:28,239 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 773391e6c583b5ff2a5d0746351f3fed#family1#compaction#49 average throughput is 17.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:28,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_] to archive 2023-02-09 14:54:28,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:28,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d5cf126b2a87465f9fb7fb89ce3fe41c 2023-02-09 14:54:28,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3017068a77544c30a28a0c887fee9b5c 2023-02-09 14:54:28,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/79fa5a5c23be43b2974b200a61d876a5 2023-02-09 14:54:28,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/8ecef094ad224673a2a09f5daa815132 2023-02-09 14:54:28,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/d09769e62aad459d841ab2fee1ce4d61 2023-02-09 14:54:28,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/066de1ac4e214d12843e504f280b62cc 2023-02-09 14:54:28,318 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 35 more 2023-02-09 14:54:28,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/5d8764c9fed143e3ba81582e91b2ea3d 2023-02-09 14:54:28,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/24ae14aaf8554b6bbc5c9dc0005c0aac 2023-02-09 14:54:28,324 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/68a91b49c1b2474ca7f26de8d1510803 2023-02-09 14:54:28,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/4db2c2a6165c4895a23ba77bece8a88b_SeqId_44_ 2023-02-09 14:54:28,326 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/.tmp/family1/dff20eb2688d436f9a123ba2e6fde252 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/dff20eb2688d436f9a123ba2e6fde252 2023-02-09 14:54:28,332 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 773391e6c583b5ff2a5d0746351f3fed/family1 of 773391e6c583b5ff2a5d0746351f3fed into dff20eb2688d436f9a123ba2e6fde252(size=166.8 K), total size for store is 166.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:28,332 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 773391e6c583b5ff2a5d0746351f3fed: 2023-02-09 14:54:28,332 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1675954468218; duration=0sec 2023-02-09 14:54:28,332 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:28,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3bb729c2f6ad4aae81f20d26978eeda1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_] to archive 2023-02-09 14:54:28,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:28,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/925513fc09d94500aebd4dc48ad0869c 2023-02-09 14:54:28,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3bb729c2f6ad4aae81f20d26978eeda1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/3bb729c2f6ad4aae81f20d26978eeda1 2023-02-09 14:54:28,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/family1/19e2f8d0e6184bd2b34d6dacce8d625f_SeqId_47_ 2023-02-09 14:54:28,375 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=252 (was 257), OpenFileDescriptor=609 (was 603) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 281), ProcessCount=168 (was 168), AvailableMemoryMB=3210 (was 3246) 2023-02-09 14:54:28,386 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=252, OpenFileDescriptor=609, MaxFileDescriptor=60000, SystemLoadAverage=237, ProcessCount=168, AvailableMemoryMB=3210 2023-02-09 14:54:28,387 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,387 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:54:28,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:28,390 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:28,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-09 14:54:28,391 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,391 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,391 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:28,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:28,392 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:28,393 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 empty. 2023-02-09 14:54:28,393 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:28,394 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-09 14:54:28,492 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,492 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:28,693 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,693 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:28,803 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:28,805 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 431c1f7d6260ecba176dba7e512abdb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:28,994 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,995 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing 431c1f7d6260ecba176dba7e512abdb1, disabling compactions & flushes 2023-02-09 14:54:29,212 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. after waiting 0 ms 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,212 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,212 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for 431c1f7d6260ecba176dba7e512abdb1: Waiting for close lock at 1675954469212Disabling compacts and flushes for region at 1675954469212Disabling writes for close at 1675954469212Writing region close event to WAL at 1675954469212Closed at 1675954469212 2023-02-09 14:54:29,215 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:29,216 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954469216"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954469216"}]},"ts":"1675954469216"} 2023-02-09 14:54:29,217 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:29,218 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:29,218 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954469218"}]},"ts":"1675954469218"} 2023-02-09 14:54:29,219 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-09 14:54:29,222 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, ASSIGN}] 2023-02-09 14:54:29,224 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, ASSIGN 2023-02-09 14:54:29,224 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:29,376 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:29,376 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954469376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954469376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954469376"}]},"ts":"1675954469376"} 2023-02-09 14:54:29,378 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:29,496 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,496 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:29,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 431c1f7d6260ecba176dba7e512abdb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:29,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:29,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,544 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,545 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:29,545 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:29,546 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 431c1f7d6260ecba176dba7e512abdb1 columnFamilyName family1 2023-02-09 14:54:29,547 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(310): Store=431c1f7d6260ecba176dba7e512abdb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:29,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:29,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:29,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 431c1f7d6260ecba176dba7e512abdb1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68526479, jitterRate=0.021124109625816345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:29,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 431c1f7d6260ecba176dba7e512abdb1: Running coprocessor pre-open hook at 1675954469538Writing region info on filesystem at 1675954469538Initializing all the Stores at 1675954469539 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954469539Cleaning up temporary data from old regions at 1675954469549 (+10 ms)Cleaning up detritus from prior splits at 1675954469549Running coprocessor post-open hooks at 1675954469554 (+5 ms)Region opened successfully at 1675954469554 2023-02-09 14:54:29,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., pid=41, masterSystemTime=1675954469533 2023-02-09 14:54:29,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:29,557 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:29,558 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954469557"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954469557"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954469557"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954469557"}]},"ts":"1675954469557"} 2023-02-09 14:54:29,560 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-09 14:54:29,561 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872 in 181 msec 2023-02-09 14:54:29,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-09 14:54:29,563 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, ASSIGN in 339 msec 2023-02-09 14:54:29,563 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:29,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954469563"}]},"ts":"1675954469563"} 2023-02-09 14:54:29,564 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-09 14:54:29,566 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:29,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1790 sec 2023-02-09 14:54:30,497 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:30,497 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:30,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 14:54:30,498 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-09 14:54:30,519 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:30,519 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:30,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:30,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,521 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:30,521 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:30,521 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:30,521 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:30,522 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,522 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:30,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:30,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:30,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:30,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:30,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:30,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:30,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:30,525 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:30,525 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:30,525 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:54:30,942 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511304ee035d6689d4f5a984f09e3d985f4f1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511304ee035d6689d4f5a984f09e3d985f4f1 2023-02-09 14:54:30,946 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/aa3bd3c0414c4d5f89166477e88b86d2 2023-02-09 14:54:30,951 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/aa3bd3c0414c4d5f89166477e88b86d2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2 2023-02-09 14:54:30,957 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-09 14:54:30,958 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 433ms, sequenceid=5, compaction requested=false 2023-02-09 14:54:30,958 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:30,958 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:30,958 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:30,958 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:30,959 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:30,960 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,960 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:30,961 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,961 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:30,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:30,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7079156a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:30,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:30,963 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,965 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:30,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:30,965 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,965 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:30,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:30,967 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:30,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:30,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:30,970 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:30,970 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:30,970 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:30,970 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,970 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:30,971 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,971 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:30,971 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:30,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:30,972 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:30,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,973 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,975 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,976 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:30,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:30,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:30,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:30,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:30,976 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:30,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:30,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:30,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:31,077 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:31,077 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,077 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:31,081 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,081 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:31,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:31,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,083 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:31,083 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:31,083 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:31,083 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:31,084 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,084 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,086 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:31,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:31,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:31,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:31,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:31,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:31,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:31,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:31,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:31,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:31,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:31,088 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:54:31,501 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112811c7cc0c65914774bb7268597da5bef4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112811c7cc0c65914774bb7268597da5bef4 2023-02-09 14:54:31,507 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/31f697615e4349b28517eea0807e7f98 2023-02-09 14:54:31,512 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/31f697615e4349b28517eea0807e7f98 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98 2023-02-09 14:54:31,517 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:54:31,518 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 430ms, sequenceid=9, compaction requested=false 2023-02-09 14:54:31,519 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:31,519 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:31,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:31,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:31,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:31,519 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:31,520 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,520 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:31,521 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,521 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21bb8669[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:31,522 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:31,522 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:31,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:31,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:31,527 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:31,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:31,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:31,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,527 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:31,530 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,530 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:31,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,530 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:31,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:31,530 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,531 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:31,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:31,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:31,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:31,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,537 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:31,537 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:31,638 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:31,638 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,638 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:31,642 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,642 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:31,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:31,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,644 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:31,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:31,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:31,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:31,645 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,645 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,646 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:31,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:31,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:31,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:31,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:31,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:31,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:31,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:31,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:31,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:31,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:31,647 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:31,647 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:31,648 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:32,059 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120120dda128f01242a4adecb75663b9d41f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120120dda128f01242a4adecb75663b9d41f 2023-02-09 14:54:32,465 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/70727d7d31a44cedad187d843af88d3b 2023-02-09 14:54:32,470 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/70727d7d31a44cedad187d843af88d3b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b 2023-02-09 14:54:32,474 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 14:54:32,475 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 827ms, sequenceid=13, compaction requested=true 2023-02-09 14:54:32,475 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:32,475 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:32,477 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:32,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:32,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:32,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45cbafc6[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:32,479 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:32,479 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,480 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:32,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:32,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:32,480 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,480 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:32,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:32,481 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:32,484 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:32,484 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:32,484 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:32,484 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,484 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:32,486 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,486 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,486 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:32,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:32,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:32,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:32,487 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,488 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:32,490 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:32,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:32,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:32,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,490 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,491 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:32,491 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:32,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:32,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:32,591 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:32,591 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:32,591 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:32,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:32,595 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:32,595 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:32,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:32,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:32,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,598 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:32,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:32,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:32,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:32,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,599 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,600 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,600 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:32,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:32,600 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:32,600 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:32,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,601 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,601 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:32,601 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:32,601 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:32,602 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:32,602 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:32,602 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:32,602 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:33,017 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120512d67af4c541450383954f401fcf679b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120512d67af4c541450383954f401fcf679b 2023-02-09 14:54:33,423 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/27a6c25387f949ecbb3d72fd18f4fa77 2023-02-09 14:54:33,427 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/27a6c25387f949ecbb3d72fd18f4fa77 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77 2023-02-09 14:54:33,431 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 14:54:33,432 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 830ms, sequenceid=17, compaction requested=true 2023-02-09 14:54:33,432 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:33,432 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:33,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:33,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:33,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:33,432 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:33,434 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,434 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26083a80[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:33,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:33,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,437 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:33,438 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:33,438 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,438 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:33,439 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:33,439 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,439 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:33,439 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:33,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:33,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:33,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:33,441 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:33,441 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:33,441 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:33,441 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,441 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:33,442 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,442 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,443 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:33,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:33,443 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,444 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:33,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:33,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:33,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:33,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:33,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:33,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:33,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:33,447 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:33,446 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:33,447 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:33,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:33,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:33,547 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:33,547 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:33,547 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:33,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:33,551 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:33,552 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:33,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:33,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:33,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,554 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:33,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:33,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:33,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:33,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,554 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,557 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:33,557 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:33,557 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:33,557 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:33,558 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:33,558 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:33,558 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:33,558 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:33,558 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:33,558 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:33,558 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:33,558 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:33,970 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122811e22650168748c8811e25604366ee5e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122811e22650168748c8811e25604366ee5e 2023-02-09 14:54:34,376 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/efda28d3d1db41dc991d3414b40ea99b 2023-02-09 14:54:34,380 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/efda28d3d1db41dc991d3414b40ea99b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b 2023-02-09 14:54:34,384 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 14:54:34,385 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 827ms, sequenceid=21, compaction requested=true 2023-02-09 14:54:34,385 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:34,385 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:34,385 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:34,385 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:34,385 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,385 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:34,385 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:34,386 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,386 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:34,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:34,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:34,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:34,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:34,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:34,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:34,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ee2b96e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:34,388 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:34,388 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,392 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:34,393 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,393 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:34,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:34,393 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:34,395 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:34,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:34,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:34,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:34,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:34,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:34,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:34,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:34,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:34,397 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:34,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:34,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:34,397 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,397 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:34,399 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,399 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,399 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:34,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:34,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:34,399 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:34,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:34,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,400 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,402 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:34,403 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:34,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:34,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:34,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:34,403 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:34,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:34,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:34,503 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:34,504 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:34,504 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:34,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:34,508 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:34,508 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:34,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:34,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:34,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,510 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:34,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:34,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:34,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:34,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,511 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:34,511 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:34,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:34,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:34,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,512 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:34,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:34,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:34,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:34,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:34,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:34,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:34,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:34,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:34,513 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:34,513 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:34,513 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:34,925 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316712ecca6dcb4ceb9ab964aae68cc3ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316712ecca6dcb4ceb9ab964aae68cc3ed 2023-02-09 14:54:35,066 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:35,332 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/cf3b933c90a5494d9c7960d20db9f6a2 2023-02-09 14:54:35,337 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/cf3b933c90a5494d9c7960d20db9f6a2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2 2023-02-09 14:54:35,340 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 14:54:35,341 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 828ms, sequenceid=25, compaction requested=true 2023-02-09 14:54:35,341 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:35,341 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:35,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:35,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:35,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:35,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:35,349 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:35,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:35,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:35,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:35,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:35,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:35,351 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:35,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6317ec15[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:35,351 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,352 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:35,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:35,352 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:35,352 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,352 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:35,353 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:35,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:35,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:35,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:35,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:35,355 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:35,355 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:35,355 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:35,355 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,355 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:35,357 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:35,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:35,357 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,358 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:35,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:35,361 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:35,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:35,361 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:35,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:35,362 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:35,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:35,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:35,462 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:35,462 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:35,462 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:35,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:35,466 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:35,466 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:35,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:35,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:35,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,468 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:35,468 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:35,468 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:35,468 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:35,469 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,469 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,470 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,470 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:35,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:35,470 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:35,470 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,471 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:35,471 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:35,471 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:35,471 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:35,471 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:35,472 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:35,472 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:35,472 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:35,883 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d664d45784ad4023892bcdf433f4b449 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d664d45784ad4023892bcdf433f4b449 2023-02-09 14:54:36,289 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/150450f44f0b479194469e767568d706 2023-02-09 14:54:36,293 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/150450f44f0b479194469e767568d706 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706 2023-02-09 14:54:36,297 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 14:54:36,298 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 826ms, sequenceid=29, compaction requested=true 2023-02-09 14:54:36,298 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:36,298 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,300 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,300 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:36,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bbada2e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:36,302 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:36,302 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,303 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:36,303 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,303 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:36,303 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:36,303 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:36,306 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:36,306 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:36,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,306 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:36,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:36,309 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:36,309 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:36,309 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:36,309 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,309 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:36,310 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,310 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,310 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:36,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:36,311 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,312 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:36,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:36,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:36,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 14:54:36,317 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:36,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:36,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:36,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,318 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,418 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:36,418 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:36,418 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:36,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:36,422 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:36,422 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:36,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:36,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:36,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,424 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:36,424 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:36,424 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:36,424 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:36,425 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,425 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,428 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,428 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:36,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:36,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:36,428 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,428 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:36,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:36,429 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:36,430 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:36,430 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:36,430 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:36,843 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103472ede6e04264c249bfb9f940e8d20ac to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103472ede6e04264c249bfb9f940e8d20ac 2023-02-09 14:54:36,848 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/7b7622115bc04fdc9c000c25596309c0 2023-02-09 14:54:36,852 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/7b7622115bc04fdc9c000c25596309c0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0 2023-02-09 14:54:36,856 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 14:54:36,857 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 427ms, sequenceid=33, compaction requested=true 2023-02-09 14:54:36,857 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:36,857 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,858 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,859 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:36,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c8123e3[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:36,860 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:36,860 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,861 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,861 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:36,861 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:36,861 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:36,861 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,861 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:36,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:36,862 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:36,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:36,865 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:36,865 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:36,865 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:36,865 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,865 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:36,869 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,869 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:36,869 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:36,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:36,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:36,869 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:36,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:36,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,870 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:36,872 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:36,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:36,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:36,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:36,872 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,873 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:36,873 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,873 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:36,973 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:36,973 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:36,973 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:36,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:36,977 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:36,977 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:36,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:36,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:36,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,979 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:36,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:36,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:36,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:36,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,981 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:36,981 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:36,981 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:36,981 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:36,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:36,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:36,982 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:36,982 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:36,982 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:36,982 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:36,982 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:36,982 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:36,982 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:36,982 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:37,394 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f85dfe73306948cc834b31a1c50f9f96 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907f85dfe73306948cc834b31a1c50f9f96 2023-02-09 14:54:37,799 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/6a8e30cbc151429c80bc5ffca2b45157 2023-02-09 14:54:37,804 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/6a8e30cbc151429c80bc5ffca2b45157 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157 2023-02-09 14:54:37,808 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 14:54:37,808 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 826ms, sequenceid=37, compaction requested=true 2023-02-09 14:54:37,809 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:37,809 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:37,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:37,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:37,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:37,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:37,810 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,810 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:37,811 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,811 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:37,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:37,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:37,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1723c0d4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:37,812 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:37,812 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,813 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,814 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:37,814 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:37,814 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:37,814 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,814 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:37,816 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:37,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:37,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:37,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:37,818 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:37,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:37,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:37,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,819 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:37,820 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,820 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:37,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,820 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:37,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:37,821 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,822 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:37,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:37,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:37,824 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:37,824 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,825 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:37,925 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:37,925 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:37,925 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:37,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:37,929 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:37,929 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:37,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:37,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:37,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,931 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:37,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:37,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 14:54:37,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 14:54:37,932 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,932 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,936 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,936 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:37,936 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:37,936 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:37,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:37,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,937 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 14:54:37,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:37,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 14:54:37,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:37,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 14:54:37,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:37,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:37,938 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:37,938 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. started... 2023-02-09 14:54:37,938 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 431c1f7d6260ecba176dba7e512abdb1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:38,351 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a27c2bd0b80640c391810055c100fed3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a27c2bd0b80640c391810055c100fed3 2023-02-09 14:54:38,757 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/50eb005222464ae98c6c0ff4857f38cb 2023-02-09 14:54:38,762 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/50eb005222464ae98c6c0ff4857f38cb as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb 2023-02-09 14:54:38,766 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 14:54:38,767 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 431c1f7d6260ecba176dba7e512abdb1 in 829ms, sequenceid=41, compaction requested=true 2023-02-09 14:54:38,767 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:38,767 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 14:54:38,767 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 14:54:38,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,769 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:38,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:38,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:38,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:38,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:38,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 14:54:38,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b08e161[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:38,770 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 14:54:38,771 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,772 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,772 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 14:54:38,772 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 14:54:38,772 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:38,772 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,772 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:38,773 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 14:54:38,773 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:38,773 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,773 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 14:54:38,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:38,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:38,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:38,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 14:54:38,776 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 14:54:38,776 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:38,776 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:38,776 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,776 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:38,781 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,781 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:38,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:38,781 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:38,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:38,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:38,782 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:38,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,783 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:38,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,785 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,785 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:38,785 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 14:54:38,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:38,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:38,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:38,785 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,785 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:38,785 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:38,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:38,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,786 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,886 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:38,886 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:38,886 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:38,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:38,888 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:38,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:38,891 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:38,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:38,891 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955078891 2023-02-09 14:54:38,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:38,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 14:54:38,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:38,895 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x75cbc747 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:38,900 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40206a4e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:38,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,737 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031a0af0e31c2f4804becd162e9811bee0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601031a0af0e31c2f4804becd162e9811bee0 2023-02-09 14:54:39,738 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:39,751 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:39,752 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34546, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:39,757 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:39,757 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:39,758 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:39,759 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50470, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:39,768 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/620791f35b7240aa90b2b00fdb0161f5 first=Optional[r3] last=Optional[r6] 2023-02-09 14:54:39,769 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/620791f35b7240aa90b2b00fdb0161f5}] 2023-02-09 14:54:39,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/620791f35b7240aa90b2b00fdb0161f5 for inclusion in 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:39,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 14:54:39,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:39,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:39,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/620791f35b7240aa90b2b00fdb0161f5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__23blaoc0ma4cr2dl3ahfln5mq3tggv1fibjdg8gcvgc8vd4n60tpf9rqt32cajid/family1/620791f35b7240aa90b2b00fdb0161f5 2023-02-09 14:54:39,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__23blaoc0ma4cr2dl3ahfln5mq3tggv1fibjdg8gcvgc8vd4n60tpf9rqt32cajid/family1/620791f35b7240aa90b2b00fdb0161f5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_ 2023-02-09 14:54:39,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__23blaoc0ma4cr2dl3ahfln5mq3tggv1fibjdg8gcvgc8vd4n60tpf9rqt32cajid/family1/620791f35b7240aa90b2b00fdb0161f5 into 431c1f7d6260ecba176dba7e512abdb1/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_ - updating store file list. 2023-02-09 14:54:39,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_ into 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:39,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__23blaoc0ma4cr2dl3ahfln5mq3tggv1fibjdg8gcvgc8vd4n60tpf9rqt32cajid/family1/620791f35b7240aa90b2b00fdb0161f5 into 431c1f7d6260ecba176dba7e512abdb1/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_) 2023-02-09 14:54:39,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__23blaoc0ma4cr2dl3ahfln5mq3tggv1fibjdg8gcvgc8vd4n60tpf9rqt32cajid/family1/620791f35b7240aa90b2b00fdb0161f5 2023-02-09 14:54:39,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:54:39,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:54:39,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 431c1f7d6260ecba176dba7e512abdb1/family1 is initiating minor compaction 2023-02-09 14:54:39,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2509efeb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:39,788 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 431c1f7d6260ecba176dba7e512abdb1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:39,788 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp, totalSize=50.4 K 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 31f697615e4349b28517eea0807e7f98, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 70727d7d31a44cedad187d843af88d3b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 27a6c25387f949ecbb3d72fd18f4fa77, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting efda28d3d1db41dc991d3414b40ea99b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting cf3b933c90a5494d9c7960d20db9f6a2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 150450f44f0b479194469e767568d706, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 7b7622115bc04fdc9c000c25596309c0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 6a8e30cbc151429c80bc5ffca2b45157, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 50eb005222464ae98c6c0ff4857f38cb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:54:39,788 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting e7450c4299e245a181bfec011515625a_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:54:39,789 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:39,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:39,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122811e22650168748c8811e25604366ee5e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122811e22650168748c8811e25604366ee5e 2023-02-09 14:54:39,793 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316712ecca6dcb4ceb9ab964aae68cc3ed to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512316712ecca6dcb4ceb9ab964aae68cc3ed 2023-02-09 14:54:39,794 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d664d45784ad4023892bcdf433f4b449 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d664d45784ad4023892bcdf433f4b449 2023-02-09 14:54:39,795 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103472ede6e04264c249bfb9f940e8d20ac to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103472ede6e04264c249bfb9f940e8d20ac 2023-02-09 14:54:39,795 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-09 14:54:39,795 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,796 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:39,814 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 431c1f7d6260ecba176dba7e512abdb1#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:54:40,630 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/800e64cb96ef45598958b6ba8a83d085 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/800e64cb96ef45598958b6ba8a83d085 2023-02-09 14:54:40,635 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 431c1f7d6260ecba176dba7e512abdb1/family1 of 431c1f7d6260ecba176dba7e512abdb1 into 800e64cb96ef45598958b6ba8a83d085(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:40,635 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:40,635 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675954479787; duration=0sec 2023-02-09 14:54:40,636 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:40,640 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fcfb698bba234e80ba5cc007e123c666 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fcfb698bba234e80ba5cc007e123c666 2023-02-09 14:54:40,641 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:40,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_] to archive 2023-02-09 14:54:40,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:40,648 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/31f697615e4349b28517eea0807e7f98 2023-02-09 14:54:40,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/70727d7d31a44cedad187d843af88d3b 2023-02-09 14:54:40,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/27a6c25387f949ecbb3d72fd18f4fa77 2023-02-09 14:54:40,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/efda28d3d1db41dc991d3414b40ea99b 2023-02-09 14:54:40,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/cf3b933c90a5494d9c7960d20db9f6a2 2023-02-09 14:54:40,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/150450f44f0b479194469e767568d706 2023-02-09 14:54:40,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/7b7622115bc04fdc9c000c25596309c0 2023-02-09 14:54:40,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/6a8e30cbc151429c80bc5ffca2b45157 2023-02-09 14:54:40,659 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/50eb005222464ae98c6c0ff4857f38cb 2023-02-09 14:54:40,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e7450c4299e245a181bfec011515625a_SeqId_44_ 2023-02-09 14:54:40,661 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:40,661 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:40,668 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/078b0d0f4e1b4f038e073e46f93b773d first=Optional[r00] last=Optional[r2] 2023-02-09 14:54:40,668 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/078b0d0f4e1b4f038e073e46f93b773d}] 2023-02-09 14:54:40,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/078b0d0f4e1b4f038e073e46f93b773d for inclusion in 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:40,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 14:54:40,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:40,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:40,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/078b0d0f4e1b4f038e073e46f93b773d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sjqtbuvb464440v6kt3cgtmkhtfd7q3lgr61od0chht66jj2klro16s5rh06s9n4/family1/078b0d0f4e1b4f038e073e46f93b773d 2023-02-09 14:54:40,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sjqtbuvb464440v6kt3cgtmkhtfd7q3lgr61od0chht66jj2klro16s5rh06s9n4/family1/078b0d0f4e1b4f038e073e46f93b773d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_ 2023-02-09 14:54:40,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sjqtbuvb464440v6kt3cgtmkhtfd7q3lgr61od0chht66jj2klro16s5rh06s9n4/family1/078b0d0f4e1b4f038e073e46f93b773d into 431c1f7d6260ecba176dba7e512abdb1/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_ - updating store file list. 2023-02-09 14:54:40,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_ into 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:40,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sjqtbuvb464440v6kt3cgtmkhtfd7q3lgr61od0chht66jj2klro16s5rh06s9n4/family1/078b0d0f4e1b4f038e073e46f93b773d into 431c1f7d6260ecba176dba7e512abdb1/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_) 2023-02-09 14:54:40,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__sjqtbuvb464440v6kt3cgtmkhtfd7q3lgr61od0chht66jj2klro16s5rh06s9n4/family1/078b0d0f4e1b4f038e073e46f93b773d 2023-02-09 14:54:40,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:54:40,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:54:40,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 431c1f7d6260ecba176dba7e512abdb1/family1 is initiating minor compaction (all files) 2023-02-09 14:54:40,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@550edb27; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:40,683 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 431c1f7d6260ecba176dba7e512abdb1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:40,683 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/800e64cb96ef45598958b6ba8a83d085, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp, totalSize=390.1 K 2023-02-09 14:54:40,684 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting aa3bd3c0414c4d5f89166477e88b86d2, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:54:40,684 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 800e64cb96ef45598958b6ba8a83d085, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:54:40,685 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840463 2023-02-09 14:54:40,685 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:40,686 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:40,688 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511304ee035d6689d4f5a984f09e3d985f4f1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511304ee035d6689d4f5a984f09e3d985f4f1 2023-02-09 14:54:40,689 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120120dda128f01242a4adecb75663b9d41f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120120dda128f01242a4adecb75663b9d41f 2023-02-09 14:54:40,691 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120512d67af4c541450383954f401fcf679b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120512d67af4c541450383954f401fcf679b 2023-02-09 14:54:40,691 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-09 14:54:40,691 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 14:54:40,694 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:40,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:40,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:40,695 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:40,697 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8060 sec 2023-02-09 14:54:40,704 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 431c1f7d6260ecba176dba7e512abdb1#family1#compaction#61 average throughput is 30.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:54:40,770 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:40,770 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:40,771 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 14:54:40,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:41,181 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:41,182 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:41,182 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:41,184 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 14:54:41,187 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE}] 2023-02-09 14:54:41,187 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE 2023-02-09 14:54:41,188 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:41,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954481188"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954481188"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954481188"}]},"ts":"1675954481188"} 2023-02-09 14:54:41,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:41,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:41,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 431c1f7d6260ecba176dba7e512abdb1, disabling compactions & flushes 2023-02-09 14:54:41,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:41,926 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/.tmp/family1/586ae33886454f5780a339b103e40411 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/586ae33886454f5780a339b103e40411 2023-02-09 14:54:41,930 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 431c1f7d6260ecba176dba7e512abdb1/family1 of 431c1f7d6260ecba176dba7e512abdb1 into 586ae33886454f5780a339b103e40411(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:54:41,931 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:41,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:41,931 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1675954480683; duration=1sec 2023-02-09 14:54:41,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:41,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. after waiting 0 ms 2023-02-09 14:54:41,931 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:41,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:41,931 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/800e64cb96ef45598958b6ba8a83d085, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_] to archive 2023-02-09 14:54:41,932 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:41,933 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/aa3bd3c0414c4d5f89166477e88b86d2 2023-02-09 14:54:41,934 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/800e64cb96ef45598958b6ba8a83d085 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/800e64cb96ef45598958b6ba8a83d085 2023-02-09 14:54:41,935 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/01e505ef76d946e2be5b7759fc19dc5b_SeqId_47_ 2023-02-09 14:54:41,939 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 14:54:41,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:41,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 431c1f7d6260ecba176dba7e512abdb1: Waiting for close lock at 1675954481341Running coprocessor pre-close hooks at 1675954481341Disabling compacts and flushes for region at 1675954481341Disabling writes for close at 1675954481931 (+590 ms)Writing region close event to WAL at 1675954481936 (+5 ms)Running coprocessor post-close hooks at 1675954481939 (+3 ms)Closed at 1675954481940 (+1 ms) 2023-02-09 14:54:41,940 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 431c1f7d6260ecba176dba7e512abdb1 to self. 2023-02-09 14:54:41,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:41,942 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=CLOSED 2023-02-09 14:54:41,942 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954481942"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954481942"}]},"ts":"1675954481942"} 2023-02-09 14:54:41,944 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 14:54:41,944 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872 in 754 msec 2023-02-09 14:54:41,945 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=true 2023-02-09 14:54:42,095 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:42,095 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954482095"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954482095"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954482095"}]},"ts":"1675954482095"} 2023-02-09 14:54:42,097 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:42,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:42,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 431c1f7d6260ecba176dba7e512abdb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:42,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:42,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,253 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,254 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:42,254 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:42,254 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 431c1f7d6260ecba176dba7e512abdb1 columnFamilyName family1 2023-02-09 14:54:42,258 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/586ae33886454f5780a339b103e40411 2023-02-09 14:54:42,258 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(310): Store=431c1f7d6260ecba176dba7e512abdb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:42,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:42,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 431c1f7d6260ecba176dba7e512abdb1; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59771064, jitterRate=-0.10934174060821533}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:42,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 431c1f7d6260ecba176dba7e512abdb1: Running coprocessor pre-open hook at 1675954482252Writing region info on filesystem at 1675954482252Initializing all the Stores at 1675954482252Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675954482252Cleaning up temporary data from old regions at 1675954482260 (+8 ms)Cleaning up detritus from prior splits at 1675954482261 (+1 ms)Running coprocessor post-open hooks at 1675954482263 (+2 ms)Region opened successfully at 1675954482263 2023-02-09 14:54:42,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., pid=47, masterSystemTime=1675954482248 2023-02-09 14:54:42,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:42,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:42,265 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:42,266 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954482265"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954482265"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954482265"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954482265"}]},"ts":"1675954482265"} 2023-02-09 14:54:42,268 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-09 14:54:42,268 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872 in 170 msec 2023-02-09 14:54:42,269 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 14:54:42,269 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE in 1.0810 sec 2023-02-09 14:54:42,271 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-09 14:54:42,271 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0860 sec 2023-02-09 14:54:42,273 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5010 sec 2023-02-09 14:54:42,273 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:42,273 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:42,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-09 14:54:42,274 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:42,274 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:42,274 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:42,275 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:42,275 INFO [Listener at localhost/33189] compactions.TestMobCompactor(278): alter status finished 2023-02-09 14:54:42,276 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:42,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:42,278 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:42,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:42,278 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955082278 2023-02-09 14:54:42,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:42,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 14:54:42,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:42,280 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x2e4a3693 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:42,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d7b4b0e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:42,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:42,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:42,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:42,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:42,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,111 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fa91290cb9764e9b9b98a993f81af99c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128fa91290cb9764e9b9b98a993f81af99c 2023-02-09 14:54:43,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:43,124 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:43,125 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34562, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:43,131 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,131 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,131 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:43,133 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50482, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:43,141 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36c38cd702104775883286700c5c107d first=Optional[r05] last=Optional[r8] 2023-02-09 14:54:43,141 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36c38cd702104775883286700c5c107d}] 2023-02-09 14:54:43,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36c38cd702104775883286700c5c107d for inclusion in 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:43,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 14:54:43,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:43,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 431c1f7d6260ecba176dba7e512abdb1: 2023-02-09 14:54:43,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36c38cd702104775883286700c5c107d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pjh4tgg3athivsitpnnp05r36ivfitfcaddgfb8kvl0ls1nimosujnhnkjg0barn/family1/36c38cd702104775883286700c5c107d 2023-02-09 14:54:43,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pjh4tgg3athivsitpnnp05r36ivfitfcaddgfb8kvl0ls1nimosujnhnkjg0barn/family1/36c38cd702104775883286700c5c107d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e00a465ab0914c089ee14cc4ece51c5c_SeqId_53_ 2023-02-09 14:54:43,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pjh4tgg3athivsitpnnp05r36ivfitfcaddgfb8kvl0ls1nimosujnhnkjg0barn/family1/36c38cd702104775883286700c5c107d into 431c1f7d6260ecba176dba7e512abdb1/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e00a465ab0914c089ee14cc4ece51c5c_SeqId_53_ - updating store file list. 2023-02-09 14:54:43,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e00a465ab0914c089ee14cc4ece51c5c_SeqId_53_ into 431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:43,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pjh4tgg3athivsitpnnp05r36ivfitfcaddgfb8kvl0ls1nimosujnhnkjg0barn/family1/36c38cd702104775883286700c5c107d into 431c1f7d6260ecba176dba7e512abdb1/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e00a465ab0914c089ee14cc4ece51c5c_SeqId_53_) 2023-02-09 14:54:43,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pjh4tgg3athivsitpnnp05r36ivfitfcaddgfb8kvl0ls1nimosujnhnkjg0barn/family1/36c38cd702104775883286700c5c107d 2023-02-09 14:54:43,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:43,158 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:43,159 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a27c2bd0b80640c391810055c100fed3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a27c2bd0b80640c391810055c100fed3 2023-02-09 14:54:43,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112811c7cc0c65914774bb7268597da5bef4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112811c7cc0c65914774bb7268597da5bef4 2023-02-09 14:54:43,160 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-09 14:54:43,160 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 14:54:43,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,161 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:43,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:43,162 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:43,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 886 msec 2023-02-09 14:54:43,218 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,218 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:54:43,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:43,229 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 14:54:43,230 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:43,230 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 14:54:43,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 14:54:43,236 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE}] 2023-02-09 14:54:43,237 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE 2023-02-09 14:54:43,237 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:43,238 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954483237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954483237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954483237"}]},"ts":"1675954483237"} 2023-02-09 14:54:43,239 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:43,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 431c1f7d6260ecba176dba7e512abdb1, disabling compactions & flushes 2023-02-09 14:54:43,391 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. after waiting 0 ms 2023-02-09 14:54:43,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 14:54:43,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 431c1f7d6260ecba176dba7e512abdb1: Waiting for close lock at 1675954483391Running coprocessor pre-close hooks at 1675954483391Disabling compacts and flushes for region at 1675954483391Disabling writes for close at 1675954483391Writing region close event to WAL at 1675954483392 (+1 ms)Running coprocessor post-close hooks at 1675954483395 (+3 ms)Closed at 1675954483396 (+1 ms) 2023-02-09 14:54:43,396 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 431c1f7d6260ecba176dba7e512abdb1 to self. 2023-02-09 14:54:43,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,398 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=CLOSED 2023-02-09 14:54:43,398 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954483398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954483398"}]},"ts":"1675954483398"} 2023-02-09 14:54:43,400 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-09 14:54:43,400 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872 in 160 msec 2023-02-09 14:54:43,401 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=true 2023-02-09 14:54:43,551 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:43,551 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954483551"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954483551"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954483551"}]},"ts":"1675954483551"} 2023-02-09 14:54:43,553 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:43,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 431c1f7d6260ecba176dba7e512abdb1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:43,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,709 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,710 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:43,710 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1 2023-02-09 14:54:43,710 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 431c1f7d6260ecba176dba7e512abdb1 columnFamilyName family1 2023-02-09 14:54:43,715 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/586ae33886454f5780a339b103e40411 2023-02-09 14:54:43,718 DEBUG [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/family1/e00a465ab0914c089ee14cc4ece51c5c_SeqId_53_ 2023-02-09 14:54:43,718 INFO [StoreOpener-431c1f7d6260ecba176dba7e512abdb1-1] regionserver.HStore(310): Store=431c1f7d6260ecba176dba7e512abdb1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:43,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:54:43,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 431c1f7d6260ecba176dba7e512abdb1; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59297101, jitterRate=-0.11640433967113495}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:43,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 431c1f7d6260ecba176dba7e512abdb1: Running coprocessor pre-open hook at 1675954483708Writing region info on filesystem at 1675954483708Initializing all the Stores at 1675954483708Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954483708Cleaning up temporary data from old regions at 1675954483721 (+13 ms)Cleaning up detritus from prior splits at 1675954483721Running coprocessor post-open hooks at 1675954483724 (+3 ms)Region opened successfully at 1675954483724 2023-02-09 14:54:43,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1., pid=53, masterSystemTime=1675954483704 2023-02-09 14:54:43,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:54:43,726 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=431c1f7d6260ecba176dba7e512abdb1, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:43,726 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675954483726"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954483726"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954483726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954483726"}]},"ts":"1675954483726"} 2023-02-09 14:54:43,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-09 14:54:43,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 431c1f7d6260ecba176dba7e512abdb1, server=jenkins-hbase4.apache.org,43455,1675954416872 in 175 msec 2023-02-09 14:54:43,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 14:54:43,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=431c1f7d6260ecba176dba7e512abdb1, REOPEN/MOVE in 493 msec 2023-02-09 14:54:43,732 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 14:54:43,732 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 499 msec 2023-02-09 14:54:43,734 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 514 msec 2023-02-09 14:54:43,734 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,735 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 14:54:43,735 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,735 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,736 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,736 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,737 INFO [Listener at localhost/33189] compactions.TestMobCompactor(278): alter status finished 2023-02-09 14:54:43,738 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 14:54:43,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:43,740 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:43,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:43,741 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955083740 2023-02-09 14:54:43,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:43,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 14:54:43,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:43,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x140c51a4 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:43,747 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23185b10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 14:54:43,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 14:54:43,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:43,749 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 14:54:43,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 11 msec 2023-02-09 14:54:43,798 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=248 (was 252), OpenFileDescriptor=628 (was 609) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=212 (was 237), ProcessCount=168 (was 168), AvailableMemoryMB=3143 (was 3210) 2023-02-09 14:54:43,808 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=248, OpenFileDescriptor=628, MaxFileDescriptor=60000, SystemLoadAverage=212, ProcessCount=168, AvailableMemoryMB=3143 2023-02-09 14:54:43,809 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,809 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,810 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 14:54:43,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:43,813 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:43,813 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-09 14:54:43,813 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,813 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,813 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:43,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:43,814 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:43,815 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 empty. 2023-02-09 14:54:43,815 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:43,815 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-09 14:54:43,914 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:43,914 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:43,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:44,115 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:44,115 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:44,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:44,223 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:44,224 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 97b6b9df89d0f2a426e3ed381519a6e4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:54:44,416 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:44,416 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:44,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:44,629 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:44,629 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 97b6b9df89d0f2a426e3ed381519a6e4, disabling compactions & flushes 2023-02-09 14:54:44,630 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,630 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,630 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. after waiting 0 ms 2023-02-09 14:54:44,630 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,630 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,630 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 97b6b9df89d0f2a426e3ed381519a6e4: Waiting for close lock at 1675954484629Disabling compacts and flushes for region at 1675954484629Disabling writes for close at 1675954484630 (+1 ms)Writing region close event to WAL at 1675954484630Closed at 1675954484630 2023-02-09 14:54:44,632 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:44,633 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954484633"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954484633"}]},"ts":"1675954484633"} 2023-02-09 14:54:44,634 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:44,634 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:44,635 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954484635"}]},"ts":"1675954484635"} 2023-02-09 14:54:44,635 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-09 14:54:44,639 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, ASSIGN}] 2023-02-09 14:54:44,640 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, ASSIGN 2023-02-09 14:54:44,641 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:54:44,792 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:44,792 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954484792"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954484792"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954484792"}]},"ts":"1675954484792"} 2023-02-09 14:54:44,794 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:44,917 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:44,918 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:44,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:44,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 97b6b9df89d0f2a426e3ed381519a6e4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:44,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:44,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,949 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,950 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:44,950 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:44,951 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97b6b9df89d0f2a426e3ed381519a6e4 columnFamilyName family1 2023-02-09 14:54:44,952 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(310): Store=97b6b9df89d0f2a426e3ed381519a6e4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:44,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:44,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:44,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 97b6b9df89d0f2a426e3ed381519a6e4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70834135, jitterRate=0.05551086366176605}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:44,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 97b6b9df89d0f2a426e3ed381519a6e4: Running coprocessor pre-open hook at 1675954484948Writing region info on filesystem at 1675954484948Initializing all the Stores at 1675954484949 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675954484949Cleaning up temporary data from old regions at 1675954484953 (+4 ms)Cleaning up detritus from prior splits at 1675954484954 (+1 ms)Running coprocessor post-open hooks at 1675954484958 (+4 ms)Region opened successfully at 1675954484958 2023-02-09 14:54:44,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., pid=57, masterSystemTime=1675954484945 2023-02-09 14:54:44,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:44,960 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:44,960 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954484960"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954484960"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954484960"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954484960"}]},"ts":"1675954484960"} 2023-02-09 14:54:44,962 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 14:54:44,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 167 msec 2023-02-09 14:54:44,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-09 14:54:44,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, ASSIGN in 324 msec 2023-02-09 14:54:44,964 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:44,964 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954484964"}]},"ts":"1675954484964"} 2023-02-09 14:54:44,965 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-09 14:54:44,967 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:44,968 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1570 sec 2023-02-09 14:54:45,918 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:45,919 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:45,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 14:54:45,920 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-09 14:54:45,937 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:45,937 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:45,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:45,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,940 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:45,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:45,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:45,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:45,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:45,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:45,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:45,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:45,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:45,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:45,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:45,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:45,944 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:45,944 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:45,944 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:45,945 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:45,945 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:45,945 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:45,945 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:45,946 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:54:45,968 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302c187300cc8b463e81c4a14254500910 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c187300cc8b463e81c4a14254500910 2023-02-09 14:54:46,372 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/6a01a3a919be420bb665517580a7d0a2 2023-02-09 14:54:46,377 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/6a01a3a919be420bb665517580a7d0a2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2 2023-02-09 14:54:46,380 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-09 14:54:46,381 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 435ms, sequenceid=5, compaction requested=false 2023-02-09 14:54:46,381 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:46,382 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:46,382 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:46,382 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:46,384 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,384 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:46,384 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,384 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:46,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:46,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:46,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@664e3984[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:46,385 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:46,385 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,387 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:46,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:46,387 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:46,387 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,388 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:46,389 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:46,391 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:46,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:46,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:46,391 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,391 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:46,393 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,393 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:46,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,393 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:46,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:46,393 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:46,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:46,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:46,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,394 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:46,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,400 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:46,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:46,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:46,400 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:46,401 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:46,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:46,401 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:46,501 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:46,502 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:46,502 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:46,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:46,506 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:46,506 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:46,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:46,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:46,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,509 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:46,509 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:46,509 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:46,509 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:46,509 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,509 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,510 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,511 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:46,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:46,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:46,511 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:46,511 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:46,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,511 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,512 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,512 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:46,512 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:46,512 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:46,513 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:54:46,925 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f052a028aa4e4ad69349288ee3c1cb94 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f052a028aa4e4ad69349288ee3c1cb94 2023-02-09 14:54:47,331 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/01eeffa8496f4537a7f703fd59067f3c 2023-02-09 14:54:47,336 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/01eeffa8496f4537a7f703fd59067f3c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c 2023-02-09 14:54:47,340 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:54:47,340 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 827ms, sequenceid=9, compaction requested=false 2023-02-09 14:54:47,341 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:47,341 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:47,341 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:47,342 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:47,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:47,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:47,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73c58095[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:47,344 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:47,344 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,345 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:47,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:47,345 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:47,345 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,345 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:47,347 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:47,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:47,347 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:47,349 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:47,349 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:47,349 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:47,349 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,349 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:47,351 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,351 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:47,351 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:47,351 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,352 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,354 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,354 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:47,354 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,355 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:47,355 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:47,355 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:47,354 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:47,355 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:47,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:47,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:47,455 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:47,455 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:47,455 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:47,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:47,459 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:47,459 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:47,460 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:47,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:47,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,461 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:47,461 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:47,461 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:47,461 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:47,462 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,462 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,463 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,463 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:47,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:47,463 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:47,463 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,464 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,464 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,464 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:47,464 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:47,464 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:47,465 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:47,465 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:47,476 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:47,542 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x140c51a4 to 127.0.0.1:56347 2023-02-09 14:54:47,542 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:47,543 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:47,543 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e4a3693 to 127.0.0.1:56347 2023-02-09 14:54:47,543 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:47,544 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:47,544 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x75cbc747 to 127.0.0.1:56347 2023-02-09 14:54:47,544 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:47,545 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:47,546 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2b501a7a to 127.0.0.1:56347 2023-02-09 14:54:47,546 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:47,548 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:54:47,548 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f954bdf to 127.0.0.1:56347 2023-02-09 14:54:47,548 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:54:47,877 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c5452bc2100e4f4995201973810655e6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5452bc2100e4f4995201973810655e6 2023-02-09 14:54:48,282 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/8083b8e5c0674aa8adf20a9def3ed2f8 2023-02-09 14:54:48,287 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/8083b8e5c0674aa8adf20a9def3ed2f8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8 2023-02-09 14:54:48,290 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 14:54:48,291 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 826ms, sequenceid=13, compaction requested=true 2023-02-09 14:54:48,291 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:48,291 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:48,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:48,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:48,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:48,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:48,293 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,293 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:48,293 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,293 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:48,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:48,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a69db59[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:48,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:48,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,296 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:48,296 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,296 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:48,297 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:48,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:48,300 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:48,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:48,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:48,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,300 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:48,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:48,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:48,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:48,303 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,304 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:48,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:48,306 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:48,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:48,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:48,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:48,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,307 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,307 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,307 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,307 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:48,307 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:48,407 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:48,407 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:48,407 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:48,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:48,411 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:48,411 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:48,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:48,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:48,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,413 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:48,413 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:48,413 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:48,413 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:48,414 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,414 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,415 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:48,415 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:48,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:48,415 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:48,415 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:48,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,416 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,416 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:48,416 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:48,416 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:48,416 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:48,416 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:48,417 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:48,417 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:48,417 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:48,417 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:48,417 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:48,829 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057941a87426f94bc4aef057f172db3742 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512057941a87426f94bc4aef057f172db3742 2023-02-09 14:54:49,235 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/ffdf85934d37412ab2e2ddca879b45e1 2023-02-09 14:54:49,239 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/ffdf85934d37412ab2e2ddca879b45e1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1 2023-02-09 14:54:49,243 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 14:54:49,244 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 827ms, sequenceid=17, compaction requested=true 2023-02-09 14:54:49,244 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:49,244 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:49,245 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:49,245 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:49,245 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,245 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:49,245 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:49,254 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,254 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:49,255 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,255 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:49,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:49,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76e63e9c[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:49,256 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:49,256 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,257 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:49,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:49,257 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,257 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:49,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:49,258 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:49,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:49,258 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:49,259 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:49,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:49,260 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:49,260 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:49,260 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:49,260 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,260 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:49,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:49,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:49,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:49,262 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,263 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:49,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:49,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,266 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,266 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:49,267 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,267 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:49,267 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:49,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:49,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:49,367 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:49,367 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:49,367 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:49,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:49,371 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:49,371 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:49,371 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:49,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:49,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,373 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:49,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:49,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:49,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:49,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,373 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,374 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:49,375 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:49,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:49,375 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:49,375 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:49,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:49,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,376 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:49,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:49,376 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:49,376 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:49,376 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:49,376 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:49,376 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:49,376 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:49,376 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:49,376 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:49,788 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c08ddfe8280846c6b173e8b0a929243e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c08ddfe8280846c6b173e8b0a929243e 2023-02-09 14:54:50,193 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/0383409782d04f30b623aa71493d7b4d 2023-02-09 14:54:50,198 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/0383409782d04f30b623aa71493d7b4d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d 2023-02-09 14:54:50,201 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 14:54:50,202 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 826ms, sequenceid=21, compaction requested=true 2023-02-09 14:54:50,202 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:50,202 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:50,202 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:50,202 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:50,202 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,202 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:50,202 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:50,204 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,204 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:50,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:50,204 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,204 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:50,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:50,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61f3bbdd[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:50,205 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:50,206 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,207 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,207 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:50,207 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:50,207 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:50,207 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,207 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:50,209 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,209 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:50,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:50,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:50,209 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:50,209 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:50,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:50,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:50,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:50,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:50,211 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:50,211 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:50,211 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:50,211 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,211 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:50,213 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,213 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:50,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,213 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:50,213 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:50,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:50,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,215 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,216 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:50,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:50,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,216 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:50,217 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:50,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:50,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:50,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:50,217 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:50,217 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:50,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:50,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:50,317 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:50,317 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:50,317 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:50,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:50,321 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:50,321 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:50,321 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:50,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:50,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,323 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:50,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:50,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:50,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:50,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,325 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:50,325 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:50,326 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:50,326 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:50,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:50,327 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:50,327 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:50,327 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:50,327 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:50,327 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:50,327 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:50,327 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:50,327 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:50,738 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123125a55393fa3946a896531c49bbcbef8a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123125a55393fa3946a896531c49bbcbef8a 2023-02-09 14:54:51,144 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/abecb47217b64e84b8f1fca8e7b5305a 2023-02-09 14:54:51,148 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/abecb47217b64e84b8f1fca8e7b5305a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a 2023-02-09 14:54:51,152 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 14:54:51,152 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 825ms, sequenceid=25, compaction requested=true 2023-02-09 14:54:51,153 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:51,153 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:51,153 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:51,153 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:51,153 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,153 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:51,153 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:51,155 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,155 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:51,156 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,156 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:51,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:51,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eed5e5e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:51,157 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:51,157 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,158 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,158 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:51,158 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:51,158 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:51,158 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,158 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:51,159 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:51,159 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:51,159 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,159 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:51,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:51,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:51,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:51,162 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:51,162 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:51,162 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:51,162 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,162 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:51,163 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,163 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:51,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:51,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,163 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:51,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:51,164 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:51,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:51,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:51,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,165 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:51,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:51,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:51,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:51,170 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:51,170 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:51,170 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:51,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:51,170 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,171 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:51,171 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,171 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,271 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:51,271 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:51,271 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:51,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:51,274 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:51,275 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:51,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:51,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:51,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,277 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:51,277 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:51,277 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:51,277 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:51,277 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,277 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,286 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:51,286 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:51,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:51,286 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:51,286 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:51,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:51,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:51,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:51,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:51,287 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:51,287 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:51,287 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:51,698 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101994b0fb968094697bdf2bfd9cda3e34b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101994b0fb968094697bdf2bfd9cda3e34b 2023-02-09 14:54:52,103 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/cc1f0d73847d40758554b4cce7d23335 2023-02-09 14:54:52,108 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/cc1f0d73847d40758554b4cce7d23335 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335 2023-02-09 14:54:52,112 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 14:54:52,113 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 826ms, sequenceid=29, compaction requested=true 2023-02-09 14:54:52,113 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:52,113 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:52,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:52,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:52,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:52,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:52,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:52,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:52,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:52,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:52,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:52,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:52,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:52,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a6ae9c9[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:52,116 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:52,116 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,117 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,117 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:52,118 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:52,118 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:52,118 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,118 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:52,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:52,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:52,120 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:52,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:52,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:52,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:52,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:52,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:52,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:52,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:52,122 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:52,122 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:52,122 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:52,122 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,122 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:52,123 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,123 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:52,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,124 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,124 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:52,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:52,125 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:52,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:52,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:52,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:52,127 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,127 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:52,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:52,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:52,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,128 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:52,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,228 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:52,228 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:52,228 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:52,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:52,232 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:52,232 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:52,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:52,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:52,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,235 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:52,235 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:52,235 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:52,235 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:52,235 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,235 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,237 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:52,237 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:52,237 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:52,237 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:52,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:52,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:52,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:52,239 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:52,239 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:52,239 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:52,239 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:52,656 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f6f74e756cbf4576af3cdb313ee67fe5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6f74e756cbf4576af3cdb313ee67fe5 2023-02-09 14:54:53,062 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a 2023-02-09 14:54:53,066 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a 2023-02-09 14:54:53,071 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 14:54:53,072 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 833ms, sequenceid=33, compaction requested=true 2023-02-09 14:54:53,072 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:53,072 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:53,072 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:53,072 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:53,072 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,072 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:53,072 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:53,074 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,074 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:53,074 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,074 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:53,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:53,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:53,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dbf34c6[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:53,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:53,076 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,077 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,077 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:53,077 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:53,077 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:53,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:53,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:53,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:53,078 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,078 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:53,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:53,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:53,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:53,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:53,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:53,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:53,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:53,081 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:53,081 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:53,081 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:53,081 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,081 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:53,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:53,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:53,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:53,085 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:53,086 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:53,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:53,088 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,089 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:53,089 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:53,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:53,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:53,089 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:53,189 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:53,190 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:53,190 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:53,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:53,194 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:53,194 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:53,194 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:53,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:53,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,197 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:53,197 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:53,197 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:53,197 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:53,197 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,197 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,199 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:53,199 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:53,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:53,199 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:53,199 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:53,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,199 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:53,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:53,200 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:53,200 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:53,200 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:53,200 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:53,200 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:53,200 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:53,201 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:53,201 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:53,613 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090712501df173174517bfc622ad2486ddbe to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015090712501df173174517bfc622ad2486ddbe 2023-02-09 14:54:54,018 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/dcf782e41c1a424bacfc019189131718 2023-02-09 14:54:54,022 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/dcf782e41c1a424bacfc019189131718 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718 2023-02-09 14:54:54,025 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 14:54:54,026 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 825ms, sequenceid=37, compaction requested=true 2023-02-09 14:54:54,026 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:54,026 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:54,026 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:54,026 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:54,026 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,026 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:54,027 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:54,029 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,029 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,029 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,029 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:54,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:54,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@316e43ff[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:54,030 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:54,030 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,031 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,031 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:54,031 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:54,031 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:54,032 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,032 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:54,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:54,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:54,033 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,033 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:54,035 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:54,035 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:54,035 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:54,035 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,035 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:54,036 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,036 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,036 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:54,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:54,037 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,038 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,041 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:54,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:54,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:54,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:54,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:54,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:54,041 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:54,141 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:54,142 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:54,142 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:54,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:54,145 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:54,145 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:54,146 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:54,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:54,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,147 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:54,147 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:54,147 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 14:54:54,147 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 14:54:54,148 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,148 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,149 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,149 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:54,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:54,149 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,149 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,150 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 14:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,150 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 14:54:54,150 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:54,150 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 14:54:54,150 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:54,150 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:54,150 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:54,151 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. started... 2023-02-09 14:54:54,151 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 97b6b9df89d0f2a426e3ed381519a6e4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:54:54,562 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c48bd9c808e54a36b66c3f222b3a09b1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c48bd9c808e54a36b66c3f222b3a09b1 2023-02-09 14:54:54,967 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/87e2f861d0214e0a9115e0b593555ef0 2023-02-09 14:54:54,972 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/87e2f861d0214e0a9115e0b593555ef0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0 2023-02-09 14:54:54,975 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 14:54:54,976 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 97b6b9df89d0f2a426e3ed381519a6e4 in 825ms, sequenceid=41, compaction requested=true 2023-02-09 14:54:54,976 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:54,976 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:54,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:54,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:54,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 14:54:54,976 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 14:54:54,978 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,978 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,978 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,978 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 14:54:54,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65a5e422[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:54,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 14:54:54,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,980 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,980 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 14:54:54,980 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 14:54:54,980 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:54:54,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,980 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 14:54:54,983 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 14:54:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:54:54,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 14:54:54,985 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 14:54:54,985 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:54,985 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:54,985 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,985 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:54,986 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,986 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,987 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:54,987 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,988 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:54,990 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:54,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:54,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 14:54:54,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 14:54:54,990 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:54,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:55,090 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:55,091 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:55,091 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:55,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:55,092 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:55,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:55,095 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:55,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:55,095 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955095095 2023-02-09 14:54:55,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:55,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 14:54:55,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:55,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x523b2ecf to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:55,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f6410bb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:55,103 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,942 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010380710efac1be4746a6fb7e3316a8354d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010380710efac1be4746a6fb7e3316a8354d 2023-02-09 14:54:55,943 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:55,956 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:55,957 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53852, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:55,963 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:55,963 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:55,963 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:55,964 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34078, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:55,972 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ff2b7e9674734dd29d87e50658502c4a first=Optional[r3] last=Optional[r6] 2023-02-09 14:54:55,973 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ff2b7e9674734dd29d87e50658502c4a}] 2023-02-09 14:54:55,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ff2b7e9674734dd29d87e50658502c4a for inclusion in 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:55,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 14:54:55,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:55,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:55,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ff2b7e9674734dd29d87e50658502c4a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jtqmdb85u0m5ovv72j8abcik3072ldalvds6klhol6sjfcvn604act4qns885k56/family1/ff2b7e9674734dd29d87e50658502c4a 2023-02-09 14:54:55,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jtqmdb85u0m5ovv72j8abcik3072ldalvds6klhol6sjfcvn604act4qns885k56/family1/ff2b7e9674734dd29d87e50658502c4a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_ 2023-02-09 14:54:55,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jtqmdb85u0m5ovv72j8abcik3072ldalvds6klhol6sjfcvn604act4qns885k56/family1/ff2b7e9674734dd29d87e50658502c4a into 97b6b9df89d0f2a426e3ed381519a6e4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_ - updating store file list. 2023-02-09 14:54:55,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_ into 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:55,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jtqmdb85u0m5ovv72j8abcik3072ldalvds6klhol6sjfcvn604act4qns885k56/family1/ff2b7e9674734dd29d87e50658502c4a into 97b6b9df89d0f2a426e3ed381519a6e4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_) 2023-02-09 14:54:55,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jtqmdb85u0m5ovv72j8abcik3072ldalvds6klhol6sjfcvn604act4qns885k56/family1/ff2b7e9674734dd29d87e50658502c4a 2023-02-09 14:54:55,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:54:55,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:54:55,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 97b6b9df89d0f2a426e3ed381519a6e4/family1 is initiating minor compaction 2023-02-09 14:54:55,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7733d580; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:55,989 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 97b6b9df89d0f2a426e3ed381519a6e4/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:55,989 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp, totalSize=50.3 K 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 01eeffa8496f4537a7f703fd59067f3c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8083b8e5c0674aa8adf20a9def3ed2f8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting ffdf85934d37412ab2e2ddca879b45e1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 0383409782d04f30b623aa71493d7b4d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting abecb47217b64e84b8f1fca8e7b5305a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting cc1f0d73847d40758554b4cce7d23335, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting c5f73f2091e34fe3a1f0cb3c9b74a37a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting dcf782e41c1a424bacfc019189131718, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 87e2f861d0214e0a9115e0b593555ef0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:54:55,989 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 6fce20d91c0a415cae9cc827722f7204_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:54:55,991 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:55,991 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:55,993 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c08ddfe8280846c6b173e8b0a929243e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228c08ddfe8280846c6b173e8b0a929243e 2023-02-09 14:54:55,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123125a55393fa3946a896531c49bbcbef8a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123125a55393fa3946a896531c49bbcbef8a 2023-02-09 14:54:55,995 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101994b0fb968094697bdf2bfd9cda3e34b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101994b0fb968094697bdf2bfd9cda3e34b 2023-02-09 14:54:55,996 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6f74e756cbf4576af3cdb313ee67fe5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6f74e756cbf4576af3cdb313ee67fe5 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-09 14:54:55,997 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,997 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:55,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:56,023 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 97b6b9df89d0f2a426e3ed381519a6e4#family1#compaction#72 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:54:56,060 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/4677272373eb4198b102b5ce92f12586 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/4677272373eb4198b102b5ce92f12586 2023-02-09 14:54:56,064 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 97b6b9df89d0f2a426e3ed381519a6e4/family1 of 97b6b9df89d0f2a426e3ed381519a6e4 into 4677272373eb4198b102b5ce92f12586(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:56,065 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:56,065 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675954495989; duration=0sec 2023-02-09 14:54:56,065 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:56,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_] to archive 2023-02-09 14:54:56,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:56,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/01eeffa8496f4537a7f703fd59067f3c 2023-02-09 14:54:56,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/8083b8e5c0674aa8adf20a9def3ed2f8 2023-02-09 14:54:56,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/ffdf85934d37412ab2e2ddca879b45e1 2023-02-09 14:54:56,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/0383409782d04f30b623aa71493d7b4d 2023-02-09 14:54:56,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/abecb47217b64e84b8f1fca8e7b5305a 2023-02-09 14:54:56,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/cc1f0d73847d40758554b4cce7d23335 2023-02-09 14:54:56,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/c5f73f2091e34fe3a1f0cb3c9b74a37a 2023-02-09 14:54:56,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dcf782e41c1a424bacfc019189131718 2023-02-09 14:54:56,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/87e2f861d0214e0a9115e0b593555ef0 2023-02-09 14:54:56,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6fce20d91c0a415cae9cc827722f7204_SeqId_44_ 2023-02-09 14:54:56,866 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fc13608a7999439893106b7a65e0bd56 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205fc13608a7999439893106b7a65e0bd56 2023-02-09 14:54:56,867 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:56,882 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:56,883 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:56,890 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0fa3d5328a98461a87acc87210dd2862 first=Optional[r00] last=Optional[r2] 2023-02-09 14:54:56,890 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0fa3d5328a98461a87acc87210dd2862}] 2023-02-09 14:54:56,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0fa3d5328a98461a87acc87210dd2862 for inclusion in 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:56,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 14:54:56,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:56,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:56,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0fa3d5328a98461a87acc87210dd2862 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ev7bkjj7j70jqafb60pemmrida8rgj8jqiv2ld2gem5dgd3pu1qla1va6oijvbtk/family1/0fa3d5328a98461a87acc87210dd2862 2023-02-09 14:54:56,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ev7bkjj7j70jqafb60pemmrida8rgj8jqiv2ld2gem5dgd3pu1qla1va6oijvbtk/family1/0fa3d5328a98461a87acc87210dd2862 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_ 2023-02-09 14:54:56,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ev7bkjj7j70jqafb60pemmrida8rgj8jqiv2ld2gem5dgd3pu1qla1va6oijvbtk/family1/0fa3d5328a98461a87acc87210dd2862 into 97b6b9df89d0f2a426e3ed381519a6e4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_ - updating store file list. 2023-02-09 14:54:56,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_ into 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:56,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ev7bkjj7j70jqafb60pemmrida8rgj8jqiv2ld2gem5dgd3pu1qla1va6oijvbtk/family1/0fa3d5328a98461a87acc87210dd2862 into 97b6b9df89d0f2a426e3ed381519a6e4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_) 2023-02-09 14:54:56,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ev7bkjj7j70jqafb60pemmrida8rgj8jqiv2ld2gem5dgd3pu1qla1va6oijvbtk/family1/0fa3d5328a98461a87acc87210dd2862 2023-02-09 14:54:56,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:54:56,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:54:56,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 97b6b9df89d0f2a426e3ed381519a6e4/family1 is initiating minor compaction (all files) 2023-02-09 14:54:56,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@600f41cb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:56,905 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 97b6b9df89d0f2a426e3ed381519a6e4/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:56,905 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/4677272373eb4198b102b5ce92f12586, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp, totalSize=388.1 K 2023-02-09 14:54:56,905 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 6a01a3a919be420bb665517580a7d0a2, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:54:56,905 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4677272373eb4198b102b5ce92f12586, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:54:56,906 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840463 2023-02-09 14:54:56,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:56,907 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:56,908 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c187300cc8b463e81c4a14254500910 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511302c187300cc8b463e81c4a14254500910 2023-02-09 14:54:56,909 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5452bc2100e4f4995201973810655e6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c5452bc2100e4f4995201973810655e6 2023-02-09 14:54:56,910 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512057941a87426f94bc4aef057f172db3742 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512057941a87426f94bc4aef057f172db3742 2023-02-09 14:54:56,910 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-09 14:54:56,910 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 14:54:56,911 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:56,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:56,911 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:56,912 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:56,914 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8190 sec 2023-02-09 14:54:56,922 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 97b6b9df89d0f2a426e3ed381519a6e4#family1#compaction#73 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:54:56,976 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:56,976 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:56,977 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 14:54:56,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:57,386 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:57,387 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:57,387 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:57,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 14:54:57,391 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE}] 2023-02-09 14:54:57,392 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE 2023-02-09 14:54:57,392 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:57,393 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954497392"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954497392"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954497392"}]},"ts":"1675954497392"} 2023-02-09 14:54:57,394 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:57,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:57,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 97b6b9df89d0f2a426e3ed381519a6e4, disabling compactions & flushes 2023-02-09 14:54:57,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,139 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/.tmp/family1/213fc0a2de5f4bb78eb097952deb5398 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/213fc0a2de5f4bb78eb097952deb5398 2023-02-09 14:54:58,143 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 97b6b9df89d0f2a426e3ed381519a6e4/family1 of 97b6b9df89d0f2a426e3ed381519a6e4 into 213fc0a2de5f4bb78eb097952deb5398(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:54:58,144 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:58,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,144 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1675954496905; duration=1sec 2023-02-09 14:54:58,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. after waiting 0 ms 2023-02-09 14:54:58,144 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:58,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,144 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/4677272373eb4198b102b5ce92f12586, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_] to archive 2023-02-09 14:54:58,145 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:58,146 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/6a01a3a919be420bb665517580a7d0a2 2023-02-09 14:54:58,147 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/4677272373eb4198b102b5ce92f12586 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/4677272373eb4198b102b5ce92f12586 2023-02-09 14:54:58,148 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/dbf234a41cc549fd8e09e9da5c285b85_SeqId_47_ 2023-02-09 14:54:58,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 14:54:58,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 97b6b9df89d0f2a426e3ed381519a6e4: Waiting for close lock at 1675954497545Running coprocessor pre-close hooks at 1675954497545Disabling compacts and flushes for region at 1675954497545Disabling writes for close at 1675954498144 (+599 ms)Writing region close event to WAL at 1675954498149 (+5 ms)Running coprocessor post-close hooks at 1675954498152 (+3 ms)Closed at 1675954498152 2023-02-09 14:54:58,152 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 97b6b9df89d0f2a426e3ed381519a6e4 to self. 2023-02-09 14:54:58,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,154 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=CLOSED 2023-02-09 14:54:58,154 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954498154"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954498154"}]},"ts":"1675954498154"} 2023-02-09 14:54:58,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-09 14:54:58,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 761 msec 2023-02-09 14:54:58,157 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=true 2023-02-09 14:54:58,307 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:58,307 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954498307"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954498307"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954498307"}]},"ts":"1675954498307"} 2023-02-09 14:54:58,308 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:58,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 97b6b9df89d0f2a426e3ed381519a6e4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:58,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:58,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,464 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,465 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:58,465 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:58,465 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97b6b9df89d0f2a426e3ed381519a6e4 columnFamilyName family1 2023-02-09 14:54:58,469 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/213fc0a2de5f4bb78eb097952deb5398 2023-02-09 14:54:58,470 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(310): Store=97b6b9df89d0f2a426e3ed381519a6e4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:58,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:58,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 97b6b9df89d0f2a426e3ed381519a6e4; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68052375, jitterRate=0.014059409499168396}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:58,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 97b6b9df89d0f2a426e3ed381519a6e4: Running coprocessor pre-open hook at 1675954498463Writing region info on filesystem at 1675954498463Initializing all the Stores at 1675954498464 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675954498464Cleaning up temporary data from old regions at 1675954498471 (+7 ms)Cleaning up detritus from prior splits at 1675954498472 (+1 ms)Running coprocessor post-open hooks at 1675954498474 (+2 ms)Region opened successfully at 1675954498474 2023-02-09 14:54:58,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., pid=63, masterSystemTime=1675954498460 2023-02-09 14:54:58,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:58,476 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:58,476 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954498476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954498476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954498476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954498476"}]},"ts":"1675954498476"} 2023-02-09 14:54:58,478 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-09 14:54:58,478 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 169 msec 2023-02-09 14:54:58,480 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-09 14:54:58,480 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE in 1.0870 sec 2023-02-09 14:54:58,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-09 14:54:58,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0910 sec 2023-02-09 14:54:58,483 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5050 sec 2023-02-09 14:54:58,483 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:58,483 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:58,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-09 14:54:58,484 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:58,484 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:58,485 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:58,485 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:58,485 INFO [Listener at localhost/33189] compactions.TestMobCompactor(278): alter status finished 2023-02-09 14:54:58,486 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:58,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:58,488 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:58,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:58,488 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955098488 2023-02-09 14:54:58,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:54:58,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 14:54:58,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:58,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x1de41312 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:58,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70c849f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:58,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:58,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:58,495 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:58,495 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:58,495 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:59,318 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a30f0bdd7afa4373b53abcc0e40a4fa8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128a30f0bdd7afa4373b53abcc0e40a4fa8 2023-02-09 14:54:59,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:59,331 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:59,333 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53858, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:59,339 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:59,339 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:59,339 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:59,341 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34090, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:59,348 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a19b4d0d5a594bba846c9c5c3802f851 first=Optional[r05] last=Optional[r8] 2023-02-09 14:54:59,349 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=51 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a19b4d0d5a594bba846c9c5c3802f851}] 2023-02-09 14:54:59,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a19b4d0d5a594bba846c9c5c3802f851 for inclusion in 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 14:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 97b6b9df89d0f2a426e3ed381519a6e4: 2023-02-09 14:54:59,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a19b4d0d5a594bba846c9c5c3802f851 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bq0csbmq9v77memqe6cf497nfms0ntsq7itqs9nmhtsjn0o7qdj6niu2s8023i45/family1/a19b4d0d5a594bba846c9c5c3802f851 2023-02-09 14:54:59,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bq0csbmq9v77memqe6cf497nfms0ntsq7itqs9nmhtsjn0o7qdj6niu2s8023i45/family1/a19b4d0d5a594bba846c9c5c3802f851 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/a1cbfc5127b944d882c384cecc0dd8ac_SeqId_53_ 2023-02-09 14:54:59,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bq0csbmq9v77memqe6cf497nfms0ntsq7itqs9nmhtsjn0o7qdj6niu2s8023i45/family1/a19b4d0d5a594bba846c9c5c3802f851 into 97b6b9df89d0f2a426e3ed381519a6e4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/a1cbfc5127b944d882c384cecc0dd8ac_SeqId_53_ - updating store file list. 2023-02-09 14:54:59,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/a1cbfc5127b944d882c384cecc0dd8ac_SeqId_53_ into 97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:54:59,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bq0csbmq9v77memqe6cf497nfms0ntsq7itqs9nmhtsjn0o7qdj6niu2s8023i45/family1/a19b4d0d5a594bba846c9c5c3802f851 into 97b6b9df89d0f2a426e3ed381519a6e4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/a1cbfc5127b944d882c384cecc0dd8ac_SeqId_53_) 2023-02-09 14:54:59,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__bq0csbmq9v77memqe6cf497nfms0ntsq7itqs9nmhtsjn0o7qdj6niu2s8023i45/family1/a19b4d0d5a594bba846c9c5c3802f851 2023-02-09 14:54:59,363 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:59,364 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:59,365 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c48bd9c808e54a36b66c3f222b3a09b1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c48bd9c808e54a36b66c3f222b3a09b1 2023-02-09 14:54:59,366 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f052a028aa4e4ad69349288ee3c1cb94 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f052a028aa4e4ad69349288ee3c1cb94 2023-02-09 14:54:59,366 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-09 14:54:59,366 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 14:54:59,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:59,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:59,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:54:59,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:54:59,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:54:59,368 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:54:59,369 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 881 msec 2023-02-09 14:54:59,416 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:59,416 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:59,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-09 14:54:59,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:54:59,827 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 14:54:59,827 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:59,828 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 14:54:59,829 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 14:54:59,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE}] 2023-02-09 14:54:59,832 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE 2023-02-09 14:54:59,832 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:54:59,832 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954499832"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954499832"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954499832"}]},"ts":"1675954499832"} 2023-02-09 14:54:59,834 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:54:59,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:59,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 97b6b9df89d0f2a426e3ed381519a6e4, disabling compactions & flushes 2023-02-09 14:54:59,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:59,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:59,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. after waiting 0 ms 2023-02-09 14:54:59,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:59,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 14:54:59,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:54:59,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 97b6b9df89d0f2a426e3ed381519a6e4: Waiting for close lock at 1675954499985Running coprocessor pre-close hooks at 1675954499985Disabling compacts and flushes for region at 1675954499985Disabling writes for close at 1675954499985Writing region close event to WAL at 1675954499986 (+1 ms)Running coprocessor post-close hooks at 1675954499989 (+3 ms)Closed at 1675954499989 2023-02-09 14:54:59,989 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 97b6b9df89d0f2a426e3ed381519a6e4 to self. 2023-02-09 14:54:59,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:54:59,991 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=CLOSED 2023-02-09 14:54:59,991 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954499991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954499991"}]},"ts":"1675954499991"} 2023-02-09 14:54:59,993 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-09 14:54:59,993 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 158 msec 2023-02-09 14:54:59,993 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=true 2023-02-09 14:55:00,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:00,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954500143"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954500143"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954500143"}]},"ts":"1675954500143"} 2023-02-09 14:55:00,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:00,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:55:00,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 97b6b9df89d0f2a426e3ed381519a6e4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:00,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:00,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,300 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,301 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:55:00,301 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1 2023-02-09 14:55:00,302 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97b6b9df89d0f2a426e3ed381519a6e4 columnFamilyName family1 2023-02-09 14:55:00,306 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/213fc0a2de5f4bb78eb097952deb5398 2023-02-09 14:55:00,309 DEBUG [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/family1/a1cbfc5127b944d882c384cecc0dd8ac_SeqId_53_ 2023-02-09 14:55:00,309 INFO [StoreOpener-97b6b9df89d0f2a426e3ed381519a6e4-1] regionserver.HStore(310): Store=97b6b9df89d0f2a426e3ed381519a6e4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:00,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:55:00,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 97b6b9df89d0f2a426e3ed381519a6e4; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64997592, jitterRate=-0.03146040439605713}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:55:00,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 97b6b9df89d0f2a426e3ed381519a6e4: Running coprocessor pre-open hook at 1675954500300Writing region info on filesystem at 1675954500300Initializing all the Stores at 1675954500300Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1675954500300Cleaning up temporary data from old regions at 1675954500311 (+11 ms)Cleaning up detritus from prior splits at 1675954500312 (+1 ms)Running coprocessor post-open hooks at 1675954500314 (+2 ms)Region opened successfully at 1675954500314 2023-02-09 14:55:00,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4., pid=69, masterSystemTime=1675954500297 2023-02-09 14:55:00,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:55:00,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:55:00,316 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=97b6b9df89d0f2a426e3ed381519a6e4, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:00,316 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675954500316"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954500316"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954500316"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954500316"}]},"ts":"1675954500316"} 2023-02-09 14:55:00,318 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-09 14:55:00,318 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 97b6b9df89d0f2a426e3ed381519a6e4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 172 msec 2023-02-09 14:55:00,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-09 14:55:00,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=97b6b9df89d0f2a426e3ed381519a6e4, REOPEN/MOVE in 487 msec 2023-02-09 14:55:00,321 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 14:55:00,321 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 491 msec 2023-02-09 14:55:00,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 904 msec 2023-02-09 14:55:00,322 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,322 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-09 14:55:00,323 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,323 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,323 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,323 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,324 INFO [Listener at localhost/33189] compactions.TestMobCompactor(278): alter status finished 2023-02-09 14:55:00,324 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 14:55:00,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:55:00,326 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:55:00,326 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:55:00,326 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955100326 2023-02-09 14:55:00,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:55:00,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 14:55:00,327 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:55:00,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x6a59512c to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:00,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58343f5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 14:55:00,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 14:55:00,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:55:00,333 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 14:55:00,333 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 8 msec 2023-02-09 14:55:00,376 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242 (was 248), OpenFileDescriptor=620 (was 628), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=152 (was 212), ProcessCount=168 (was 168), AvailableMemoryMB=3065 (was 3143) 2023-02-09 14:55:00,387 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=242, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=152, ProcessCount=168, AvailableMemoryMB=3065 2023-02-09 14:55:00,388 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,388 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 14:55:00,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:00,391 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:00,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-09 14:55:00,391 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,391 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:00,391 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:00,393 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:00,393 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb empty. 2023-02-09 14:55:00,393 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:00,393 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 14:55:00,492 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,492 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:00,693 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,693 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:00,800 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:00,802 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => a6c4f65664b00bb26234b540afd74afb, NAME => 'testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:00,994 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:00,994 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:00,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing a6c4f65664b00bb26234b540afd74afb, disabling compactions & flushes 2023-02-09 14:55:01,207 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. after waiting 0 ms 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,207 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,207 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for a6c4f65664b00bb26234b540afd74afb: Waiting for close lock at 1675954501207Disabling compacts and flushes for region at 1675954501207Disabling writes for close at 1675954501207Writing region close event to WAL at 1675954501207Closed at 1675954501207 2023-02-09 14:55:01,209 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:01,210 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954501210"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954501210"}]},"ts":"1675954501210"} 2023-02-09 14:55:01,211 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:01,212 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:01,212 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954501212"}]},"ts":"1675954501212"} 2023-02-09 14:55:01,213 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-09 14:55:01,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, ASSIGN}] 2023-02-09 14:55:01,218 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, ASSIGN 2023-02-09 14:55:01,218 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:01,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=a6c4f65664b00bb26234b540afd74afb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:01,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954501369"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954501369"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954501369"}]},"ts":"1675954501369"} 2023-02-09 14:55:01,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure a6c4f65664b00bb26234b540afd74afb, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:01,495 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:01,495 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:01,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:01,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6c4f65664b00bb26234b540afd74afb, NAME => 'testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:01,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-09 14:55:01,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-09 14:55:01,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:01,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,528 INFO [StoreOpener-a6c4f65664b00bb26234b540afd74afb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,529 DEBUG [StoreOpener-a6c4f65664b00bb26234b540afd74afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1 2023-02-09 14:55:01,529 DEBUG [StoreOpener-a6c4f65664b00bb26234b540afd74afb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1 2023-02-09 14:55:01,529 INFO [StoreOpener-a6c4f65664b00bb26234b540afd74afb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6c4f65664b00bb26234b540afd74afb columnFamilyName f1 2023-02-09 14:55:01,530 INFO [StoreOpener-a6c4f65664b00bb26234b540afd74afb-1] regionserver.HStore(310): Store=a6c4f65664b00bb26234b540afd74afb/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:01,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:01,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:01,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a6c4f65664b00bb26234b540afd74afb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66381260, jitterRate=-0.01084214448928833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:55:01,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a6c4f65664b00bb26234b540afd74afb: Running coprocessor pre-open hook at 1675954501527Writing region info on filesystem at 1675954501527Initializing all the Stores at 1675954501527Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954501527Cleaning up temporary data from old regions at 1675954501532 (+5 ms)Cleaning up detritus from prior splits at 1675954501532Running coprocessor post-open hooks at 1675954501535 (+3 ms)Region opened successfully at 1675954501535 2023-02-09 14:55:01,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb., pid=73, masterSystemTime=1675954501522 2023-02-09 14:55:01,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:01,538 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=a6c4f65664b00bb26234b540afd74afb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:01,538 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954501538"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954501538"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954501538"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954501538"}]},"ts":"1675954501538"} 2023-02-09 14:55:01,541 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 14:55:01,541 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure a6c4f65664b00bb26234b540afd74afb, server=jenkins-hbase4.apache.org,43455,1675954416872 in 168 msec 2023-02-09 14:55:01,542 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-09 14:55:01,542 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, ASSIGN in 325 msec 2023-02-09 14:55:01,543 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:01,543 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954501543"}]},"ts":"1675954501543"} 2023-02-09 14:55:01,544 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-09 14:55:01,546 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:01,547 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1570 sec 2023-02-09 14:55:02,400 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-09 14:55:02,496 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:02,496 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:02,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 14:55:02,497 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-09 14:55:02,497 DEBUG [Listener at localhost/33189] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-09 14:55:02,499 INFO [Listener at localhost/33189] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 14:55:02,502 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-09 14:55:02,502 INFO [Listener at localhost/33189] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 14:55:02,502 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-09 14:55:02,506 INFO [Listener at localhost/33189] regionserver.HRegion(2744): Flushing a6c4f65664b00bb26234b540afd74afb 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 14:55:02,918 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209730686d88490489897977c4b08e02c23 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209730686d88490489897977c4b08e02c23 2023-02-09 14:55:03,323 INFO [Listener at localhost/33189] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp/f1/90fda25f4c024d0ea6bfa7e61beb456d 2023-02-09 14:55:03,328 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp/f1/90fda25f4c024d0ea6bfa7e61beb456d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d 2023-02-09 14:55:03,332 INFO [Listener at localhost/33189] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 14:55:03,333 INFO [Listener at localhost/33189] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for a6c4f65664b00bb26234b540afd74afb in 827ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:03,334 DEBUG [Listener at localhost/33189] regionserver.HRegion(2445): Flush status journal for a6c4f65664b00bb26234b540afd74afb: 2023-02-09 14:55:03,336 INFO [Listener at localhost/33189] regionserver.HRegion(2744): Flushing a6c4f65664b00bb26234b540afd74afb 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 14:55:03,682 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:03,747 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209dc11b80329744cb49bdd591b2a4ef3d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209dc11b80329744cb49bdd591b2a4ef3d2 2023-02-09 14:55:04,152 INFO [Listener at localhost/33189] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp/f1/301f6120624e4b4f9c5f5f1b79b84648 2023-02-09 14:55:04,156 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp/f1/301f6120624e4b4f9c5f5f1b79b84648 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648 2023-02-09 14:55:04,159 INFO [Listener at localhost/33189] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648, entries=1, sequenceid=9, filesize=4.9 K 2023-02-09 14:55:04,159 INFO [Listener at localhost/33189] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for a6c4f65664b00bb26234b540afd74afb in 823ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:04,160 DEBUG [Listener at localhost/33189] regionserver.HRegion(2445): Flush status journal for a6c4f65664b00bb26234b540afd74afb: 2023-02-09 14:55:04,161 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-09 14:55:04,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 14:55:04,163 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 14:55:04,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:55:04,163 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955104163 2023-02-09 14:55:04,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:55:04,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-09 14:55:04,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-09 14:55:04,164 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x6080aca5 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:04,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19b79100, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:04,168 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 14:55:04,212 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,212 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,263 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,263 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,314 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,314 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,364 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,364 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,415 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,415 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,466 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,466 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,517 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,517 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,567 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,568 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,618 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,618 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,669 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,669 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,720 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,720 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,770 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,771 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,821 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,821 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,872 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,872 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,923 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,923 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,974 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,974 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:04,990 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302099abc8cfdbe284f7987566251381a381f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302099abc8cfdbe284f7987566251381a381f 2023-02-09 14:55:04,991 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 14:55:05,004 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:05,005 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47206, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:05,009 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:05,010 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:05,010 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:55:05,011 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33034, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:55:05,019 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/c0b88501b6724683a4fe1314d3a45470 first=Optional[r1] last=Optional[r2] 2023-02-09 14:55:05,020 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{f1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/c0b88501b6724683a4fe1314d3a45470}] 2023-02-09 14:55:05,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/c0b88501b6724683a4fe1314d3a45470 for inclusion in a6c4f65664b00bb26234b540afd74afb/f1 2023-02-09 14:55:05,024 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,024 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 14:55:05,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:55:05,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for a6c4f65664b00bb26234b540afd74afb: 2023-02-09 14:55:05,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/c0b88501b6724683a4fe1314d3a45470 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testGetAfterCompaction__i6ncuc88nlnmmmbvbk3vn2f4b53ad8483q5vg1aqgl7u9qdqet13tvrlmq1oodf9/f1/c0b88501b6724683a4fe1314d3a45470 2023-02-09 14:55:05,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testGetAfterCompaction__i6ncuc88nlnmmmbvbk3vn2f4b53ad8483q5vg1aqgl7u9qdqet13tvrlmq1oodf9/f1/c0b88501b6724683a4fe1314d3a45470 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_ 2023-02-09 14:55:05,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testGetAfterCompaction__i6ncuc88nlnmmmbvbk3vn2f4b53ad8483q5vg1aqgl7u9qdqet13tvrlmq1oodf9/f1/c0b88501b6724683a4fe1314d3a45470 into a6c4f65664b00bb26234b540afd74afb/f1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_ - updating store file list. 2023-02-09 14:55:05,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_ into a6c4f65664b00bb26234b540afd74afb/f1 2023-02-09 14:55:05,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testGetAfterCompaction__i6ncuc88nlnmmmbvbk3vn2f4b53ad8483q5vg1aqgl7u9qdqet13tvrlmq1oodf9/f1/c0b88501b6724683a4fe1314d3a45470 into a6c4f65664b00bb26234b540afd74afb/f1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_) 2023-02-09 14:55:05,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testGetAfterCompaction__i6ncuc88nlnmmmbvbk3vn2f4b53ad8483q5vg1aqgl7u9qdqet13tvrlmq1oodf9/f1/c0b88501b6724683a4fe1314d3a45470 2023-02-09 14:55:05,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): a6c4f65664b00bb26234b540afd74afb/f1 is initiating minor compaction 2023-02-09 14:55:05,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@537f21e9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:05,032 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a6c4f65664b00bb26234b540afd74afb/f1 in testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:05,032 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp, totalSize=10.0 K 2023-02-09 14:55:05,032 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 301f6120624e4b4f9c5f5f1b79b84648, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:55:05,033 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 6c8c61033806429dba94ef8d2373407d_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:55:05,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 14:55:05,035 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:05,036 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209730686d88490489897977c4b08e02c23 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209730686d88490489897977c4b08e02c23 2023-02-09 14:55:05,038 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209dc11b80329744cb49bdd591b2a4ef3d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209dc11b80329744cb49bdd591b2a4ef3d2 2023-02-09 14:55:05,038 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-09 14:55:05,038 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 14:55:05,038 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 14:55:05,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-09 14:55:05,039 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:55:05,040 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 14:55:05,040 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): a6c4f65664b00bb26234b540afd74afb#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:05,041 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 879 msec 2023-02-09 14:55:05,075 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,075 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,075 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,078 DEBUG [Listener at localhost/33189] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:55:05,082 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:55:05,082 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43455] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:05,082 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-09 14:55:05,082 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 14:55:05,083 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43455] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 14:55:05,083 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43455] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. because compaction request was cancelled 2023-02-09 14:55:05,134 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,134 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,134 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,185 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,185 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,185 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,236 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,236 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,236 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,287 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,287 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,287 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,337 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,338 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,338 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,388 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,388 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,388 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,439 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,439 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,439 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,490 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,490 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,490 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,541 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,541 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,541 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,592 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,592 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,592 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,643 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,643 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,643 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,693 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,693 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,694 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,744 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,744 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,744 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,795 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,795 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,795 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,846 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,846 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,846 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,853 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/.tmp/f1/5b432181a61c412abdca3ac00d971d1b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b 2023-02-09 14:55:05,857 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in a6c4f65664b00bb26234b540afd74afb/f1 of a6c4f65664b00bb26234b540afd74afb into 5b432181a61c412abdca3ac00d971d1b(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:05,857 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6c4f65664b00bb26234b540afd74afb: 2023-02-09 14:55:05,857 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1675954505032; duration=0sec 2023-02-09 14:55:05,858 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:05,877 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-09 14:55:05,877 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-02-09 14:55:05,897 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954504151; access_time=1675954503338; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,897 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675954505848; access_time=1675954505037; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,897 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,897 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_] to archive 2023-02-09 14:55:05,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:05,947 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/301f6120624e4b4f9c5f5f1b79b84648 2023-02-09 14:55:05,947 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675954505848; access_time=1675954505037; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,947 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675954504987; access_time=1675954504179; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 14:55:05,947 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/6c8c61033806429dba94ef8d2373407d_SeqId_12_ 2023-02-09 14:55:05,998 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675954505848; access_time=1675954505037; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:05,998 INFO [Listener at localhost/33189] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675954503323; access_time=1675954502509; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 14:55:06,114 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,114 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,115 INFO [Listener at localhost/33189] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-09 14:55:06,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-09 14:55:06,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,127 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954506127"}]},"ts":"1675954506127"} 2023-02-09 14:55:06,128 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,128 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,128 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-09 14:55:06,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 14:55:06,129 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-09 14:55:06,131 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, UNASSIGN}] 2023-02-09 14:55:06,132 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, UNASSIGN 2023-02-09 14:55:06,132 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a6c4f65664b00bb26234b540afd74afb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:06,132 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954506132"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954506132"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954506132"}]},"ts":"1675954506132"} 2023-02-09 14:55:06,134 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure a6c4f65664b00bb26234b540afd74afb, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:06,229 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,229 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 14:55:06,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:06,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a6c4f65664b00bb26234b540afd74afb, disabling compactions & flushes 2023-02-09 14:55:06,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:06,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:06,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. after waiting 0 ms 2023-02-09 14:55:06,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:06,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 14:55:06,289 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-09 14:55:06,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb. 2023-02-09 14:55:06,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a6c4f65664b00bb26234b540afd74afb: Waiting for close lock at 1675954506285Running coprocessor pre-close hooks at 1675954506285Disabling compacts and flushes for region at 1675954506285Disabling writes for close at 1675954506285Writing region close event to WAL at 1675954506286 (+1 ms)Running coprocessor post-close hooks at 1675954506289 (+3 ms)Closed at 1675954506290 (+1 ms) 2023-02-09 14:55:06,292 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:06,292 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=a6c4f65664b00bb26234b540afd74afb, regionState=CLOSED 2023-02-09 14:55:06,292 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954506292"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954506292"}]},"ts":"1675954506292"} 2023-02-09 14:55:06,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-09 14:55:06,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure a6c4f65664b00bb26234b540afd74afb, server=jenkins-hbase4.apache.org,43455,1675954416872 in 160 msec 2023-02-09 14:55:06,295 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-09 14:55:06,296 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=a6c4f65664b00bb26234b540afd74afb, UNASSIGN in 163 msec 2023-02-09 14:55:06,296 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954506296"}]},"ts":"1675954506296"} 2023-02-09 14:55:06,297 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-09 14:55:06,302 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-09 14:55:06,303 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-02-09 14:55:06,430 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,430 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 14:55:06,431 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-09 14:55:06,432 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,432 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,435 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-09 14:55:06,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,443 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,444 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,444 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,444 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 14:55:06,446 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:06,448 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1, FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/recovered.edits] 2023-02-09 14:55:06,452 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/5b432181a61c412abdca3ac00d971d1b 2023-02-09 14:55:06,453 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/f1/90fda25f4c024d0ea6bfa7e61beb456d 2023-02-09 14:55:06,455 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/recovered.edits/15.seqid to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb/recovered.edits/15.seqid 2023-02-09 14:55:06,456 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testGetAfterCompaction/a6c4f65664b00bb26234b540afd74afb 2023-02-09 14:55:06,456 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 14:55:06,456 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 14:55:06,456 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-09 14:55:06,459 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302099abc8cfdbe284f7987566251381a381f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302099abc8cfdbe284f7987566251381a381f 2023-02-09 14:55:06,459 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 14:55:06,461 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,483 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-09 14:55:06,485 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-09 14:55:06,486 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,486 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-09 14:55:06,486 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954506486"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:06,487 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 14:55:06,487 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a6c4f65664b00bb26234b540afd74afb, NAME => 'testGetAfterCompaction,,1675954500389.a6c4f65664b00bb26234b540afd74afb.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 14:55:06,487 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-09 14:55:06,487 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954506487"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:06,489 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-09 14:55:06,490 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 14:55:06,491 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 54 msec 2023-02-09 14:55:06,545 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,545 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 14:55:06,546 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-09 14:55:06,559 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245 (was 242) - Thread LEAK? -, OpenFileDescriptor=613 (was 620), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 152), ProcessCount=168 (was 168), AvailableMemoryMB=3065 (was 3065) 2023-02-09 14:55:06,569 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=245, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=139, ProcessCount=168, AvailableMemoryMB=3065 2023-02-09 14:55:06,570 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,570 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,571 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 14:55:06,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-09 14:55:06,573 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:06,573 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-09 14:55:06,574 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,574 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,574 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:06,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf empty. 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b empty. 2023-02-09 14:55:06,576 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 empty. 2023-02-09 14:55:06,577 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:06,577 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:06,577 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:06,577 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-09 14:55:06,674 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,675 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:06,876 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:06,876 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:06,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:06,985 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:06,987 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => bb3d06ff69b5e78251fb182a52cc2b7b, NAME => 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:06,987 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 45873f0ac0c88ebf31d0fcc61f3d2515, NAME => 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:06,987 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0f4e5cd8c7b1d17e86c4691d686f3baf, NAME => 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 45873f0ac0c88ebf31d0fcc61f3d2515, disabling compactions & flushes 2023-02-09 14:55:06,998 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. after waiting 0 ms 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:06,998 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:06,998 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 45873f0ac0c88ebf31d0fcc61f3d2515: Waiting for close lock at 1675954506998Disabling compacts and flushes for region at 1675954506998Disabling writes for close at 1675954506998Writing region close event to WAL at 1675954506998Closed at 1675954506998 2023-02-09 14:55:07,177 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:07,177 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:07,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing bb3d06ff69b5e78251fb182a52cc2b7b, disabling compactions & flushes 2023-02-09 14:55:07,394 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. after waiting 0 ms 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,394 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for bb3d06ff69b5e78251fb182a52cc2b7b: Waiting for close lock at 1675954507394Disabling compacts and flushes for region at 1675954507394Disabling writes for close at 1675954507394Writing region close event to WAL at 1675954507394Closed at 1675954507394 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 0f4e5cd8c7b1d17e86c4691d686f3baf, disabling compactions & flushes 2023-02-09 14:55:07,399 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. after waiting 0 ms 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,399 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,399 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: Waiting for close lock at 1675954507399Disabling compacts and flushes for region at 1675954507399Disabling writes for close at 1675954507399Writing region close event to WAL at 1675954507399Closed at 1675954507399 2023-02-09 14:55:07,401 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:07,402 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954507402"}]},"ts":"1675954507402"} 2023-02-09 14:55:07,402 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954507402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954507402"}]},"ts":"1675954507402"} 2023-02-09 14:55:07,402 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954507402"}]},"ts":"1675954507402"} 2023-02-09 14:55:07,404 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 14:55:07,405 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:07,405 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954507405"}]},"ts":"1675954507405"} 2023-02-09 14:55:07,406 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 14:55:07,411 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN}] 2023-02-09 14:55:07,412 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN 2023-02-09 14:55:07,412 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN 2023-02-09 14:55:07,412 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN 2023-02-09 14:55:07,413 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:07,413 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:07,413 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:07,564 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,564 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,564 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,564 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507564"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954507564"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954507564"}]},"ts":"1675954507564"} 2023-02-09 14:55:07,565 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507564"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954507564"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954507564"}]},"ts":"1675954507564"} 2023-02-09 14:55:07,565 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954507564"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954507564"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954507564"}]},"ts":"1675954507564"} 2023-02-09 14:55:07,566 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:07,566 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:07,568 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:07,678 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:07,678 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:07,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:07,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f4e5cd8c7b1d17e86c4691d686f3baf, NAME => 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 14:55:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,723 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,724 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:07,724 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:07,725 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f4e5cd8c7b1d17e86c4691d686f3baf columnFamilyName family1 2023-02-09 14:55:07,726 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(310): Store=0f4e5cd8c7b1d17e86c4691d686f3baf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,726 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,727 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2 2023-02-09 14:55:07,727 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2 2023-02-09 14:55:07,727 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f4e5cd8c7b1d17e86c4691d686f3baf columnFamilyName family2 2023-02-09 14:55:07,728 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(310): Store=0f4e5cd8c7b1d17e86c4691d686f3baf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:07,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:07,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:07,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0f4e5cd8c7b1d17e86c4691d686f3baf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72285086, jitterRate=0.0771317183971405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:07,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: Running coprocessor pre-open hook at 1675954507722Writing region info on filesystem at 1675954507722Initializing all the Stores at 1675954507723 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507723Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507723Cleaning up temporary data from old regions at 1675954507730 (+7 ms)Cleaning up detritus from prior splits at 1675954507730Running coprocessor post-open hooks at 1675954507734 (+4 ms)Region opened successfully at 1675954507734 2023-02-09 14:55:07,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., pid=83, masterSystemTime=1675954507719 2023-02-09 14:55:07,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:07,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:07,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 45873f0ac0c88ebf31d0fcc61f3d2515, NAME => 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 14:55:07,736 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:07,737 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507736"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954507736"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954507736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954507736"}]},"ts":"1675954507736"} 2023-02-09 14:55:07,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,738 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,739 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:07,739 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:07,739 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 14:55:07,739 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45873f0ac0c88ebf31d0fcc61f3d2515 columnFamilyName family1 2023-02-09 14:55:07,739 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872 in 172 msec 2023-02-09 14:55:07,740 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN in 329 msec 2023-02-09 14:55:07,740 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(310): Store=45873f0ac0c88ebf31d0fcc61f3d2515/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,741 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,742 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2 2023-02-09 14:55:07,742 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2 2023-02-09 14:55:07,742 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45873f0ac0c88ebf31d0fcc61f3d2515 columnFamilyName family2 2023-02-09 14:55:07,743 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(310): Store=45873f0ac0c88ebf31d0fcc61f3d2515/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:07,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:07,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:07,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 45873f0ac0c88ebf31d0fcc61f3d2515; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71641644, jitterRate=0.06754368543624878}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:07,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 45873f0ac0c88ebf31d0fcc61f3d2515: Running coprocessor pre-open hook at 1675954507737Writing region info on filesystem at 1675954507737Initializing all the Stores at 1675954507737Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507737Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507737Cleaning up temporary data from old regions at 1675954507744 (+7 ms)Cleaning up detritus from prior splits at 1675954507745 (+1 ms)Running coprocessor post-open hooks at 1675954507748 (+3 ms)Region opened successfully at 1675954507748 2023-02-09 14:55:07,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., pid=84, masterSystemTime=1675954507719 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:07,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:07,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bb3d06ff69b5e78251fb182a52cc2b7b, NAME => 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 14:55:07,750 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:07,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954507750"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954507750"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954507750"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954507750"}]},"ts":"1675954507750"} 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,751 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,753 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:07,753 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:07,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-02-09 14:55:07,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872 in 185 msec 2023-02-09 14:55:07,753 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb3d06ff69b5e78251fb182a52cc2b7b columnFamilyName family1 2023-02-09 14:55:07,754 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN in 342 msec 2023-02-09 14:55:07,754 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(310): Store=bb3d06ff69b5e78251fb182a52cc2b7b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,755 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,756 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2 2023-02-09 14:55:07,756 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2 2023-02-09 14:55:07,756 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb3d06ff69b5e78251fb182a52cc2b7b columnFamilyName family2 2023-02-09 14:55:07,757 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(310): Store=bb3d06ff69b5e78251fb182a52cc2b7b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:07,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:07,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:07,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:07,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bb3d06ff69b5e78251fb182a52cc2b7b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74767275, jitterRate=0.11411921679973602}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:07,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bb3d06ff69b5e78251fb182a52cc2b7b: Running coprocessor pre-open hook at 1675954507750Writing region info on filesystem at 1675954507750Initializing all the Stores at 1675954507751 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507751Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954507751Cleaning up temporary data from old regions at 1675954507758 (+7 ms)Cleaning up detritus from prior splits at 1675954507759 (+1 ms)Running coprocessor post-open hooks at 1675954507762 (+3 ms)Region opened successfully at 1675954507762 2023-02-09 14:55:07,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., pid=85, masterSystemTime=1675954507719 2023-02-09 14:55:07,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:07,765 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:07,765 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954507765"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954507765"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954507765"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954507765"}]},"ts":"1675954507765"} 2023-02-09 14:55:07,767 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-02-09 14:55:07,767 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872 in 199 msec 2023-02-09 14:55:07,768 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-09 14:55:07,768 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN in 357 msec 2023-02-09 14:55:07,769 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:07,769 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954507769"}]},"ts":"1675954507769"} 2023-02-09 14:55:07,770 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 14:55:07,772 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:07,773 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2010 sec 2023-02-09 14:55:08,402 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-09 14:55:08,679 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:08,679 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:08,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:55:08,680 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-09 14:55:08,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:08,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:08,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:08,691 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:08,691 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:08,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:08,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 14:55:08,693 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 14:55:08,693 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:08,693 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 14:55:08,693 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 14:55:08,694 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:08,694 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:08,695 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:08,695 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:08,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:08,695 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:08,695 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 14:55:08,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 14:55:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 14:55:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:08,696 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 14:55:08,697 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:08,697 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:08,697 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:08,697 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:08,697 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:08,697 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f4e5cd8c7b1d17e86c4691d686f3baf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:08,698 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bb3d06ff69b5e78251fb182a52cc2b7b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:08,697 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:08,697 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:08,698 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:08,698 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 45873f0ac0c88ebf31d0fcc61f3d2515 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:08,721 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee 2023-02-09 14:55:08,725 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/c737342c58ad431c8113d091ddee0a2b 2023-02-09 14:55:09,115 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 2023-02-09 14:55:09,118 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 2023-02-09 14:55:09,137 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b 2023-02-09 14:55:09,520 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/ecd1a90c9bce4d668e4db5c09795f947 2023-02-09 14:55:09,522 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/af7831ca485342db8c538339e234fdc4 2023-02-09 14:55:09,542 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/389bf55f1d4f48b3a3abe466d8e22257 2023-02-09 14:55:09,545 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/c737342c58ad431c8113d091ddee0a2b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b 2023-02-09 14:55:09,548 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:09,549 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/389bf55f1d4f48b3a3abe466d8e22257 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257 2023-02-09 14:55:09,551 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:09,552 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bb3d06ff69b5e78251fb182a52cc2b7b in 854ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:09,553 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:09,553 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:09,553 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:09,934 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106 2023-02-09 14:55:09,934 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7 2023-02-09 14:55:10,339 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/fed4b332e11d4aebb3b1feb4ac19aa74 2023-02-09 14:55:10,339 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/d3eef8aad70b4a0696d86297ddf49f38 2023-02-09 14:55:10,345 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/ecd1a90c9bce4d668e4db5c09795f947 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947 2023-02-09 14:55:10,345 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/af7831ca485342db8c538339e234fdc4 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4 2023-02-09 14:55:10,349 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:10,349 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:10,350 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/d3eef8aad70b4a0696d86297ddf49f38 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38 2023-02-09 14:55:10,350 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/fed4b332e11d4aebb3b1feb4ac19aa74 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74 2023-02-09 14:55:10,353 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:10,354 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:10,354 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 45873f0ac0c88ebf31d0fcc61f3d2515 in 1656ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:10,354 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:10,354 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:10,354 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:10,355 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 0f4e5cd8c7b1d17e86c4691d686f3baf in 1658ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:10,355 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:10,355 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 14:55:10,355 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 14:55:10,357 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,357 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:10,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:10,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 14:55:10,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@385c6dfe[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:10,359 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 14:55:10,359 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,360 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 14:55:10,360 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,360 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:10,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 14:55:10,360 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:10,363 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 14:55:10,363 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,363 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:10,363 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 14:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:10,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 14:55:10,365 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 14:55:10,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:10,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:10,365 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 14:55:10,365 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:10,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:10,366 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,367 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:10,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,368 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:10,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:10,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 14:55:10,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:10,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:10,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 14:55:10,370 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:10,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:10,370 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:10,371 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:10,371 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:10,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:10,371 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,371 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:10,371 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,471 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:10,471 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:10,471 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:10,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:10,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:10,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:10,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:10,475 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:10,475 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:10,476 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:10,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:10,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 14:55:10,478 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 14:55:10,478 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:10,478 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 14:55:10,478 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 14:55:10,479 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,479 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:10,480 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:10,480 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:10,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:10,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:10,480 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:10,480 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:10,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:10,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:10,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:10,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 14:55:10,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 14:55:10,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 14:55:10,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 14:55:10,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:10,481 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:10,482 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:10,482 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:10,482 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 45873f0ac0c88ebf31d0fcc61f3d2515 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:10,482 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:10,482 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f4e5cd8c7b1d17e86c4691d686f3baf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:10,482 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bb3d06ff69b5e78251fb182a52cc2b7b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:10,896 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 2023-02-09 14:55:10,896 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a 2023-02-09 14:55:10,896 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd 2023-02-09 14:55:11,301 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/362a4118a0014b19a7654a57ad370173 2023-02-09 14:55:11,301 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 2023-02-09 14:55:11,302 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/cd8327f927a7450a895525be078bf12a 2023-02-09 14:55:11,719 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c 2023-02-09 14:55:11,719 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1 2023-02-09 14:55:11,719 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119 2023-02-09 14:55:11,791 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:12,124 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/515a4a199e4b4a088edbdb32e2cc9ad9 2023-02-09 14:55:12,126 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/f64b69aaaf5448ebab3776de43936cf4 2023-02-09 14:55:12,129 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/362a4118a0014b19a7654a57ad370173 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173 2023-02-09 14:55:12,129 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/ec14e07f0451485ca28fb2f19704e7fd 2023-02-09 14:55:12,130 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/cd8327f927a7450a895525be078bf12a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a 2023-02-09 14:55:12,133 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:12,133 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:12,134 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 2023-02-09 14:55:12,134 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/515a4a199e4b4a088edbdb32e2cc9ad9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9 2023-02-09 14:55:12,134 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/f64b69aaaf5448ebab3776de43936cf4 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4 2023-02-09 14:55:12,139 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:12,139 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:12,139 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:12,139 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/ec14e07f0451485ca28fb2f19704e7fd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd 2023-02-09 14:55:12,139 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bb3d06ff69b5e78251fb182a52cc2b7b in 1657ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:12,140 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:12,140 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:12,140 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:12,140 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 45873f0ac0c88ebf31d0fcc61f3d2515 in 1658ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:12,140 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:12,140 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:12,140 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:12,143 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:12,144 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 0f4e5cd8c7b1d17e86c4691d686f3baf in 1662ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:12,144 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:12,144 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:12,144 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:12,144 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:12,144 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,144 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 14:55:12,144 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 14:55:12,146 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:12,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:12,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 14:55:12,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cf16737[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:12,148 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 14:55:12,148 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,149 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,149 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 14:55:12,149 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 14:55:12,149 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,149 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:12,149 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:12,151 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 14:55:12,151 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:12,151 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,151 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 14:55:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:12,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 14:55:12,153 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 14:55:12,153 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:12,153 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:12,153 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 14:55:12,153 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:12,154 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,154 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:12,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,154 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:12,155 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,156 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:12,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,159 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,159 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:12,159 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:12,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:12,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 14:55:12,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:12,159 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:12,160 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:12,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,160 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:12,260 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:12,260 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:12,261 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:12,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:12,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:12,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:12,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:12,264 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:12,264 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:12,265 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:12,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:12,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 14:55:12,267 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 14:55:12,267 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:12,267 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 14:55:12,267 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 14:55:12,267 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,267 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:12,268 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,268 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:12,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:12,269 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:12,269 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:12,269 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 14:55:12,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 14:55:12,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 14:55:12,270 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 14:55:12,270 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:12,270 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:12,270 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:12,270 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:12,270 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f4e5cd8c7b1d17e86c4691d686f3baf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:12,270 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:12,270 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bb3d06ff69b5e78251fb182a52cc2b7b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:12,270 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 45873f0ac0c88ebf31d0fcc61f3d2515 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:12,287 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 2023-02-09 14:55:12,294 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/53e3f1923aa043b4b60ae7ff7486d95a 2023-02-09 14:55:12,296 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae 2023-02-09 14:55:12,300 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/36650b407d0d483594a88d050743a70d 2023-02-09 14:55:12,306 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76 2023-02-09 14:55:12,316 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53 2023-02-09 14:55:12,688 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 2023-02-09 14:55:12,715 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/189968c135f743b2818d742b290ad7e4 2023-02-09 14:55:12,719 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/53e3f1923aa043b4b60ae7ff7486d95a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a 2023-02-09 14:55:12,719 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/5cf9cb87440b48f1acd34d12d8cf054d 2023-02-09 14:55:12,723 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:12,723 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/36650b407d0d483594a88d050743a70d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d 2023-02-09 14:55:12,724 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/189968c135f743b2818d742b290ad7e4 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4 2023-02-09 14:55:12,727 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:12,728 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/5cf9cb87440b48f1acd34d12d8cf054d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d 2023-02-09 14:55:12,734 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:12,734 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:12,734 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bb3d06ff69b5e78251fb182a52cc2b7b in 464ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:12,734 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 0f4e5cd8c7b1d17e86c4691d686f3baf in 464ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:12,735 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:12,735 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:12,735 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:12,735 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:12,735 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:12,735 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:13,092 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/3ee6ce1303934fb0934193cf812d8f92 2023-02-09 14:55:13,508 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d 2023-02-09 14:55:13,913 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/bfd9d4fc5ead40caab8aa9d276ac1773 2023-02-09 14:55:13,917 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/3ee6ce1303934fb0934193cf812d8f92 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92 2023-02-09 14:55:13,920 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:13,921 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/bfd9d4fc5ead40caab8aa9d276ac1773 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773 2023-02-09 14:55:13,924 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:13,925 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 45873f0ac0c88ebf31d0fcc61f3d2515 in 1655ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:13,925 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:13,925 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 14:55:13,925 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 14:55:13,927 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,927 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:13,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:13,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 14:55:13,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f43aced[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:13,928 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 14:55:13,928 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,931 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,931 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 14:55:13,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,931 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:13,931 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 14:55:13,931 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:13,932 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,932 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 14:55:13,932 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:13,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:13,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:13,932 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 14:55:13,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:13,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:13,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:13,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:13,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 14:55:13,934 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 14:55:13,934 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:13,934 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:13,934 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 14:55:13,934 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:13,938 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,938 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:13,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,938 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:13,939 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,940 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:13,942 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:13,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:13,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:13,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 14:55:13,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:13,942 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:13,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:13,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:13,943 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:13,943 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:14,043 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:14,043 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:14,043 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:14,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:14,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:14,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:14,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:14,047 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:14,047 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:14,047 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:14,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:14,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 14:55:14,049 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 14:55:14,049 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:14,049 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 14:55:14,049 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 14:55:14,050 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:14,050 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:14,051 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:14,051 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:14,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:14,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:14,051 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:14,052 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 14:55:14,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 14:55:14,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 14:55:14,053 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 14:55:14,053 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:14,053 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:14,053 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:14,053 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:14,053 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:14,053 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bb3d06ff69b5e78251fb182a52cc2b7b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:14,053 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f4e5cd8c7b1d17e86c4691d686f3baf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:14,053 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 45873f0ac0c88ebf31d0fcc61f3d2515 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:14,465 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e 2023-02-09 14:55:14,469 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 2023-02-09 14:55:14,469 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/de20c5a9e953467cafa10278e6044c1e 2023-02-09 14:55:14,469 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 2023-02-09 14:55:14,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/c33cf885662b4628a3cb7f31236e384a 2023-02-09 14:55:14,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/464954a5d01a4933b2b28fc2837c42ab 2023-02-09 14:55:14,885 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e 2023-02-09 14:55:14,891 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777 2023-02-09 14:55:15,290 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/a426dfd82b5a434ea24052a45c394de0 2023-02-09 14:55:15,292 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3 2023-02-09 14:55:15,294 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/de20c5a9e953467cafa10278e6044c1e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e 2023-02-09 14:55:15,296 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/566bbbea376e4418a50116016187e529 2023-02-09 14:55:15,297 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/b7bf9bdd44174d6aa109091b2cad7fe1 2023-02-09 14:55:15,298 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:15,298 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/a426dfd82b5a434ea24052a45c394de0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0 2023-02-09 14:55:15,301 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/c33cf885662b4628a3cb7f31236e384a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a 2023-02-09 14:55:15,302 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:15,303 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 45873f0ac0c88ebf31d0fcc61f3d2515 in 1250ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:15,303 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:15,303 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:15,303 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:15,305 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/464954a5d01a4933b2b28fc2837c42ab as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab 2023-02-09 14:55:15,307 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:15,308 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:15,308 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/b7bf9bdd44174d6aa109091b2cad7fe1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1 2023-02-09 14:55:15,308 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/566bbbea376e4418a50116016187e529 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529 2023-02-09 14:55:15,311 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:15,312 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:15,312 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 0f4e5cd8c7b1d17e86c4691d686f3baf in 1259ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:15,312 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:15,312 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:15,312 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bb3d06ff69b5e78251fb182a52cc2b7b in 1259ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:15,312 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:15,312 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,313 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 14:55:15,313 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 14:55:15,314 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,314 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:15,314 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,314 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 14:55:15,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58dc5d3[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:15,316 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 14:55:15,316 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,318 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 14:55:15,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 14:55:15,318 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:15,318 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,318 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 14:55:15,319 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:15,319 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 14:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:15,321 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 14:55:15,321 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:15,321 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:15,321 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 14:55:15,321 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:15,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 14:55:15,323 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:15,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:15,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:15,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:15,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:15,327 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:15,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:15,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 14:55:15,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:15,327 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:15,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 14:55:15,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:15,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:15,427 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:15,427 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:15,428 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:15,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:15,437 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:15,437 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:15,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:15,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675954515438 (current time:1675954515438). 2023-02-09 14:55:15,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 14:55:15,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] zookeeper.ReadOnlyZKClient(139): Connect 0x49594587 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:15,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@180d56aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:15,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:15,452 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46986, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:15,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49594587 to 127.0.0.1:56347 2023-02-09 14:55:15,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:15,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 14:55:15,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:15,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:15,463 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:15,463 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675955115463 2023-02-09 14:55:15,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:15,464 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1675954515428 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 14:55:15,465 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:15,465 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 14:55:15,466 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-09 14:55:15,466 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 14:55:15,467 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1675955115467 2023-02-09 14:55:15,469 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:15,469 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 14:55:15,473 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1675954515428 2023-02-09 14:55:15,473 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1675954515428' 2023-02-09 14:55:15,473 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 14:55:15,473 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1675954515428' starting 'acquire' 2023-02-09 14:55:15,473 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1675954515428', kicking off acquire phase on members. 2023-02-09 14:55:15,473 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,473 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1675954515428 2023-02-09 14:55:15,475 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 14:55:15,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 14:55:15,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:55:15,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,475 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:15,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1675954515428 2023-02-09 14:55:15,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-09 14:55:15,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1675954515428 2023-02-09 14:55:15,476 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1675954515428 from table testCompactionWithHFileLink type FLUSH 2023-02-09 14:55:15,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1675954515428 2023-02-09 14:55:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1675954515428' with timeout 300000ms 2023-02-09 14:55:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 14:55:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1675954515428' starting 'acquire' stage 2023-02-09 14:55:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1675954515428' locally acquired 2023-02-09 14:55:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (snaptb-1675954515428) in zk 2023-02-09 14:55:15,480 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 14:55:15,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,480 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1675954515428' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:15,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'snaptb-1675954515428' on coordinator 2023-02-09 14:55:15,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ee57b01[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:15,481 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1675954515428' starting 'in-barrier' execution. 2023-02-09 14:55:15,481 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,483 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,483 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1675954515428' received 'reached' from coordinator. 2023-02-09 14:55:15,483 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,483 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:15,484 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:15,484 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:15,484 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-09 14:55:15,484 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:15,484 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.' region-info for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.' region-info for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,485 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.' region-info for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,490 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 14:55:15,490 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 14:55:15,490 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a] hfiles 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab] hfiles 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e] hfiles 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1] hfiles 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529] hfiles 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0] hfiles 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,497 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0 for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. completed. 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. completed. 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. completed. 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:15,508 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1675954515428' locally completed 2023-02-09 14:55:15,509 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1675954515428' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:15,510 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1675954515428' has notified controller of completion 2023-02-09 14:55:15,511 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:15,511 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,511 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1675954515428' completed. 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1675954515428' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:15,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'snaptb-1675954515428', counting down latch. Waiting for 0 more 2023-02-09 14:55:15,513 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1675954515428' execution completed 2023-02-09 14:55:15,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:15,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:15,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1675954515428 2023-02-09 14:55:15,513 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1675954515428including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-09 14:55:15,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 14:55:15,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 14:55:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:15,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:15,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,516 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675954515428 2023-02-09 14:55:15,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675954515428 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675954515428 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 14:55:15,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:15,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675954515428/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:15,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:55:15,518 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1675954515428 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,518 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675954515428 2023-02-09 14:55:15,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 14:55:15,518 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1675954515428 2023-02-09 14:55:15,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675954515428 2023-02-09 14:55:15,518 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-09 14:55:15,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:55:15,519 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a] mob files 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae 2023-02-09 14:55:15,520 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53] mob files 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c 2023-02-09 14:55:15,521 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53 2023-02-09 14:55:15,569 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:15,569 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:15,570 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:15,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:15,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:15,573 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 14:55:15,773 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:15,773 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:15,773 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:15,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:15,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:15,774 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 14:55:15,877 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-09 14:55:15,926 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1675954515428 2023-02-09 14:55:15,928 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/snaptb-1675954515428 2023-02-09 14:55:16,075 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:16,075 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:16,075 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:16,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:16,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:16,076 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 14:55:16,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/snaptb-1675954515428 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/snaptb-1675954515428 2023-02-09 14:55:16,576 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:16,576 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:16,576 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:16,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:16,577 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 14:55:16,798 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1675954515428 of table testCompactionWithHFileLink completed 2023-02-09 14:55:16,798 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/snaptb-1675954515428 2023-02-09 14:55:16,799 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/snaptb-1675954515428 2023-02-09 14:55:16,799 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1675954515428 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675954515464Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1675954515926 (+462 ms)Consolidate snapshot: snaptb-1675954515428 at 1675954515926Loading Region manifests for snaptb-1675954515428 at 1675954515926Writing data manifest for snaptb-1675954515428 at 1675954515940 (+14 ms)Verifying snapshot: snaptb-1675954515428 at 1675954516353 (+413 ms)Snapshot snaptb-1675954515428 of table testCompactionWithHFileLink completed at 1675954516798 (+445 ms) 2023-02-09 14:55:16,800 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 14:55:16,801 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.3350 sec 2023-02-09 14:55:17,578 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:17,578 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:17,578 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:17,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:17,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 14:55:17,606 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:17,606 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:17,606 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 14:55:17,608 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 14:55:17,608 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:17,608 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 14:55:17,608 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 14:55:17,609 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:17,609 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:17,610 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:17,611 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:17,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:17,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:17,611 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:17,611 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:17,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:17,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:17,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:17,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:17,612 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 14:55:17,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 14:55:17,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 14:55:17,612 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 14:55:17,612 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:17,612 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:17,612 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. started... 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:17,612 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. started... 2023-02-09 14:55:17,612 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. started... 2023-02-09 14:55:17,613 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bb3d06ff69b5e78251fb182a52cc2b7b 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:17,612 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 45873f0ac0c88ebf31d0fcc61f3d2515 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:17,613 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f4e5cd8c7b1d17e86c4691d686f3baf 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:17,633 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/947388931b884bc7974073dc265b15a1 2023-02-09 14:55:17,636 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 947388931b884bc7974073dc265b15a1 2023-02-09 14:55:18,034 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/26cdcf80a8ad4b52a1ab0f2be939663d 2023-02-09 14:55:18,037 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26cdcf80a8ad4b52a1ab0f2be939663d 2023-02-09 14:55:18,048 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/3b16c036d37b4288b4be7cfb4da97527 2023-02-09 14:55:18,051 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b16c036d37b4288b4be7cfb4da97527 2023-02-09 14:55:18,052 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/947388931b884bc7974073dc265b15a1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1 2023-02-09 14:55:18,055 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 947388931b884bc7974073dc265b15a1 2023-02-09 14:55:18,055 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:18,055 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/3b16c036d37b4288b4be7cfb4da97527 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527 2023-02-09 14:55:18,058 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b16c036d37b4288b4be7cfb4da97527 2023-02-09 14:55:18,059 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:18,059 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bb3d06ff69b5e78251fb182a52cc2b7b in 446ms, sequenceid=24, compaction requested=true 2023-02-09 14:55:18,059 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:18,059 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:18,059 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:18,428 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/1365367b5527487abcda96f5e1a4e1d3 2023-02-09 14:55:18,432 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1365367b5527487abcda96f5e1a4e1d3 2023-02-09 14:55:18,847 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/6e09234dfc214da88c0ad9ae25bedf77 2023-02-09 14:55:18,850 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e09234dfc214da88c0ad9ae25bedf77 2023-02-09 14:55:18,851 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/26cdcf80a8ad4b52a1ab0f2be939663d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d 2023-02-09 14:55:18,854 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26cdcf80a8ad4b52a1ab0f2be939663d 2023-02-09 14:55:18,855 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:18,855 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/6e09234dfc214da88c0ad9ae25bedf77 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77 2023-02-09 14:55:18,859 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e09234dfc214da88c0ad9ae25bedf77 2023-02-09 14:55:18,859 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:18,860 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 45873f0ac0c88ebf31d0fcc61f3d2515 in 1248ms, sequenceid=24, compaction requested=true 2023-02-09 14:55:18,860 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:18,860 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:18,860 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:19,245 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/02d57bbf44ba4795a95ea2a100ebceb2 2023-02-09 14:55:19,248 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02d57bbf44ba4795a95ea2a100ebceb2 2023-02-09 14:55:19,249 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/1365367b5527487abcda96f5e1a4e1d3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3 2023-02-09 14:55:19,252 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1365367b5527487abcda96f5e1a4e1d3 2023-02-09 14:55:19,252 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:19,253 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/02d57bbf44ba4795a95ea2a100ebceb2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2 2023-02-09 14:55:19,256 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02d57bbf44ba4795a95ea2a100ebceb2 2023-02-09 14:55:19,256 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 14:55:19,257 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 0f4e5cd8c7b1d17e86c4691d686f3baf in 1644ms, sequenceid=24, compaction requested=true 2023-02-09 14:55:19,257 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:19,257 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:19,257 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,258 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 14:55:19,258 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 14:55:19,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,259 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:19,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:19,259 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,259 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:19,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:19,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:19,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 14:55:19,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59aa7735[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:19,261 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 14:55:19,261 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,263 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 14:55:19,263 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 14:55:19,263 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:19,263 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,263 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 14:55:19,267 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:19,267 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 14:55:19,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:19,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 14:55:19,269 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 14:55:19,269 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:19,269 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:19,269 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 14:55:19,269 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:19,270 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,270 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:19,270 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,271 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:19,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,272 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 14:55:19,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:19,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:19,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 14:55:19,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 14:55:19,275 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:19,275 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:19,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 14:55:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:19,276 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:19,276 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,276 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:19,276 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 14:55:19,276 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 14:55:19,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:19,376 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:19,376 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:19,376 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:19,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:19,378 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:19,378 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 is initiating major compaction (all files) 2023-02-09 14:55:19,378 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:19,378 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:19,379 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 in testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:19,379 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp, totalSize=26.7 K 2023-02-09 14:55:19,380 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting ecd1a90c9bce4d668e4db5c09795f947, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508687 2023-02-09 14:55:19,380 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting f13bcf5f89e9491db5b90e31e0bd5aa8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:19,380 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 36650b407d0d483594a88d050743a70d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512262 2023-02-09 14:55:19,381 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting c33cf885662b4628a3cb7f31236e384a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:19,381 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 1365367b5527487abcda96f5e1a4e1d3, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:19,394 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 0f4e5cd8c7b1d17e86c4691d686f3baf#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:19,802 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del 2023-02-09 14:55:19,802 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del 2023-02-09 14:55:20,615 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e187b6bd1f0546eda9ca224a94a7ae26 2023-02-09 14:55:20,616 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/e187b6bd1f0546eda9ca224a94a7ae26 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/e187b6bd1f0546eda9ca224a94a7ae26 2023-02-09 14:55:20,619 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e187b6bd1f0546eda9ca224a94a7ae26 2023-02-09 14:55:20,620 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 of 0f4e5cd8c7b1d17e86c4691d686f3baf into e187b6bd1f0546eda9ca224a94a7ae26(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:20,620 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:20,620 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:20,620 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 is initiating major compaction (all files) 2023-02-09 14:55:20,620 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:20,620 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:20,621 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 in testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:20,621 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp, totalSize=25.4 K 2023-02-09 14:55:20,622 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting fed4b332e11d4aebb3b1feb4ac19aa74, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508687 2023-02-09 14:55:20,622 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting ec14e07f0451485ca28fb2f19704e7fd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:20,622 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 5cf9cb87440b48f1acd34d12d8cf054d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512262 2023-02-09 14:55:20,623 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting b7bf9bdd44174d6aa109091b2cad7fe1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:20,623 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 02d57bbf44ba4795a95ea2a100ebceb2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:20,634 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 0f4e5cd8c7b1d17e86c4691d686f3baf#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:20,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3] to archive 2023-02-09 14:55:20,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:20,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/ecd1a90c9bce4d668e4db5c09795f947 2023-02-09 14:55:20,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/f13bcf5f89e9491db5b90e31e0bd5aa8 2023-02-09 14:55:20,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/36650b407d0d483594a88d050743a70d 2023-02-09 14:55:20,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/c33cf885662b4628a3cb7f31236e384a 2023-02-09 14:55:20,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/1365367b5527487abcda96f5e1a4e1d3 2023-02-09 14:55:21,042 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del 2023-02-09 14:55:21,042 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del 2023-02-09 14:55:21,380 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:21,855 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84135e5cb2f2433a95e61adda7d9b8da 2023-02-09 14:55:21,856 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/84135e5cb2f2433a95e61adda7d9b8da as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/84135e5cb2f2433a95e61adda7d9b8da 2023-02-09 14:55:21,860 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84135e5cb2f2433a95e61adda7d9b8da 2023-02-09 14:55:21,861 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 of 0f4e5cd8c7b1d17e86c4691d686f3baf into 84135e5cb2f2433a95e61adda7d9b8da(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:21,861 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:21,861 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:21,861 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): bb3d06ff69b5e78251fb182a52cc2b7b/family1 is initiating major compaction (all files) 2023-02-09 14:55:21,861 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:21,861 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:21,862 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of bb3d06ff69b5e78251fb182a52cc2b7b/family1 in testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:21,862 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp, totalSize=26.7 K 2023-02-09 14:55:21,863 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting c737342c58ad431c8113d091ddee0a2b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508688 2023-02-09 14:55:21,863 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting cd8327f927a7450a895525be078bf12a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:21,864 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 53e3f1923aa043b4b60ae7ff7486d95a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:21,864 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 464954a5d01a4933b2b28fc2837c42ab, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:21,864 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 947388931b884bc7974073dc265b15a1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:21,874 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): bb3d06ff69b5e78251fb182a52cc2b7b#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:21,885 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del 2023-02-09 14:55:21,885 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del 2023-02-09 14:55:21,938 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2] to archive 2023-02-09 14:55:21,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:21,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/fed4b332e11d4aebb3b1feb4ac19aa74 2023-02-09 14:55:21,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/ec14e07f0451485ca28fb2f19704e7fd 2023-02-09 14:55:21,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/5cf9cb87440b48f1acd34d12d8cf054d 2023-02-09 14:55:21,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/b7bf9bdd44174d6aa109091b2cad7fe1 2023-02-09 14:55:21,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/02d57bbf44ba4795a95ea2a100ebceb2 2023-02-09 14:55:22,698 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4378510017b0419aba2abe9bc0f93f51 2023-02-09 14:55:22,699 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/4378510017b0419aba2abe9bc0f93f51 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4378510017b0419aba2abe9bc0f93f51 2023-02-09 14:55:22,703 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4378510017b0419aba2abe9bc0f93f51 2023-02-09 14:55:22,704 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bb3d06ff69b5e78251fb182a52cc2b7b/family1 of bb3d06ff69b5e78251fb182a52cc2b7b into 4378510017b0419aba2abe9bc0f93f51(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:22,704 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:22,704 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:22,704 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): bb3d06ff69b5e78251fb182a52cc2b7b/family2 is initiating major compaction (all files) 2023-02-09 14:55:22,705 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:22,705 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:22,705 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of bb3d06ff69b5e78251fb182a52cc2b7b/family2 in testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:22,706 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp, totalSize=25.4 K 2023-02-09 14:55:22,706 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 389bf55f1d4f48b3a3abe466d8e22257, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508688 2023-02-09 14:55:22,706 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting f64b69aaaf5448ebab3776de43936cf4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:22,706 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 189968c135f743b2818d742b290ad7e4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:22,707 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 566bbbea376e4418a50116016187e529, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:22,707 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3b16c036d37b4288b4be7cfb4da97527, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:22,717 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): bb3d06ff69b5e78251fb182a52cc2b7b#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:22,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1] to archive 2023-02-09 14:55:22,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:22,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/c737342c58ad431c8113d091ddee0a2b 2023-02-09 14:55:22,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/cd8327f927a7450a895525be078bf12a 2023-02-09 14:55:22,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/53e3f1923aa043b4b60ae7ff7486d95a 2023-02-09 14:55:22,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/464954a5d01a4933b2b28fc2837c42ab 2023-02-09 14:55:22,744 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/947388931b884bc7974073dc265b15a1 2023-02-09 14:55:23,129 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del 2023-02-09 14:55:23,130 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del 2023-02-09 14:55:23,944 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4f20b605f20487789e4112a5736a023 2023-02-09 14:55:23,944 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/d4f20b605f20487789e4112a5736a023 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/d4f20b605f20487789e4112a5736a023 2023-02-09 14:55:23,948 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4f20b605f20487789e4112a5736a023 2023-02-09 14:55:23,949 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bb3d06ff69b5e78251fb182a52cc2b7b/family2 of bb3d06ff69b5e78251fb182a52cc2b7b into d4f20b605f20487789e4112a5736a023(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:23,949 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:23,949 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:23,949 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 45873f0ac0c88ebf31d0fcc61f3d2515/family1 is initiating major compaction (all files) 2023-02-09 14:55:23,949 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:23,949 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:23,950 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 45873f0ac0c88ebf31d0fcc61f3d2515/family1 in testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:23,950 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp, totalSize=26.7 K 2023-02-09 14:55:23,950 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting af7831ca485342db8c538339e234fdc4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508689 2023-02-09 14:55:23,951 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 362a4118a0014b19a7654a57ad370173, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:23,951 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3ee6ce1303934fb0934193cf812d8f92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:23,951 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting de20c5a9e953467cafa10278e6044c1e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:23,952 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 26cdcf80a8ad4b52a1ab0f2be939663d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:23,964 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 45873f0ac0c88ebf31d0fcc61f3d2515#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:24,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527] to archive 2023-02-09 14:55:24,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:24,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/389bf55f1d4f48b3a3abe466d8e22257 2023-02-09 14:55:24,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/f64b69aaaf5448ebab3776de43936cf4 2023-02-09 14:55:24,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/189968c135f743b2818d742b290ad7e4 2023-02-09 14:55:24,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/566bbbea376e4418a50116016187e529 2023-02-09 14:55:24,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/3b16c036d37b4288b4be7cfb4da97527 2023-02-09 14:55:24,373 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del 2023-02-09 14:55:24,373 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del 2023-02-09 14:55:25,186 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ca41bb69b6b4c519213ae5bac615839 2023-02-09 14:55:25,187 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/7ca41bb69b6b4c519213ae5bac615839 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/7ca41bb69b6b4c519213ae5bac615839 2023-02-09 14:55:25,191 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ca41bb69b6b4c519213ae5bac615839 2023-02-09 14:55:25,192 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 45873f0ac0c88ebf31d0fcc61f3d2515/family1 of 45873f0ac0c88ebf31d0fcc61f3d2515 into 7ca41bb69b6b4c519213ae5bac615839(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:25,192 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:25,192 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:25,192 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 45873f0ac0c88ebf31d0fcc61f3d2515/family2 is initiating major compaction (all files) 2023-02-09 14:55:25,192 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:25,192 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:25,193 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 45873f0ac0c88ebf31d0fcc61f3d2515/family2 in testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:25,193 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp, totalSize=25.4 K 2023-02-09 14:55:25,193 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting d3eef8aad70b4a0696d86297ddf49f38, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508689 2023-02-09 14:55:25,194 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 515a4a199e4b4a088edbdb32e2cc9ad9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:25,194 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting bfd9d4fc5ead40caab8aa9d276ac1773, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:25,194 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting a426dfd82b5a434ea24052a45c394de0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:25,194 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 6e09234dfc214da88c0ad9ae25bedf77, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 14:55:25,205 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 45873f0ac0c88ebf31d0fcc61f3d2515#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:25,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d] to archive 2023-02-09 14:55:25,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:25,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/af7831ca485342db8c538339e234fdc4 2023-02-09 14:55:25,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/362a4118a0014b19a7654a57ad370173 2023-02-09 14:55:25,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/3ee6ce1303934fb0934193cf812d8f92 2023-02-09 14:55:25,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/de20c5a9e953467cafa10278e6044c1e 2023-02-09 14:55:25,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/26cdcf80a8ad4b52a1ab0f2be939663d 2023-02-09 14:55:25,615 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del 2023-02-09 14:55:25,615 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del 2023-02-09 14:55:26,429 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d2440cb877e465fa091f747724714ee 2023-02-09 14:55:26,430 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/6d2440cb877e465fa091f747724714ee as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6d2440cb877e465fa091f747724714ee 2023-02-09 14:55:26,433 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d2440cb877e465fa091f747724714ee 2023-02-09 14:55:26,434 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 45873f0ac0c88ebf31d0fcc61f3d2515/family2 of 45873f0ac0c88ebf31d0fcc61f3d2515 into 6d2440cb877e465fa091f747724714ee(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:26,434 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:26,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77] to archive 2023-02-09 14:55:26,439 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:26,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:26,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/d3eef8aad70b4a0696d86297ddf49f38 2023-02-09 14:55:26,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/515a4a199e4b4a088edbdb32e2cc9ad9 2023-02-09 14:55:26,442 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:26,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/bfd9d4fc5ead40caab8aa9d276ac1773 2023-02-09 14:55:26,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/a426dfd82b5a434ea24052a45c394de0 2023-02-09 14:55:26,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6e09234dfc214da88c0ad9ae25bedf77 2023-02-09 14:55:26,448 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:26,449 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:26,455 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:55:26,488 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 14:55:26,491 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del 2023-02-09 14:55:26,494 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del 2023-02-09 14:55:26,497 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del 2023-02-09 14:55:26,497 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:26,497 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(139): Connect 0x5a432736 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:26,503 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2778e3cb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:26,504 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:26,504 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:26,504 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:26,575 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913 2023-02-09 14:55:26,576 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:26,589 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:26,593 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46846, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:26,599 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:26,599 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:26,599 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:55:26,602 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36786, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:55:26,610 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/bbc5f0f408a2451eb73bae2019e6c9b9 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 14:55:26,611 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/bbc5f0f408a2451eb73bae2019e6c9b9}] 2023-02-09 14:55:26,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/bbc5f0f408a2451eb73bae2019e6c9b9 for inclusion in 45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:26,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 14:55:26,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 14:55:26,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:26,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/bbc5f0f408a2451eb73bae2019e6c9b9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__f1qtrk1rjas7bcr7lluc0gaaih872ddrbf1o538f3ab4dec9n02tukvmp9sgtil5/family1/bbc5f0f408a2451eb73bae2019e6c9b9 2023-02-09 14:55:26,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__f1qtrk1rjas7bcr7lluc0gaaih872ddrbf1o538f3ab4dec9n02tukvmp9sgtil5/family1/bbc5f0f408a2451eb73bae2019e6c9b9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ 2023-02-09 14:55:26,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__f1qtrk1rjas7bcr7lluc0gaaih872ddrbf1o538f3ab4dec9n02tukvmp9sgtil5/family1/bbc5f0f408a2451eb73bae2019e6c9b9 into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ - updating store file list. 2023-02-09 14:55:26,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:26,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__f1qtrk1rjas7bcr7lluc0gaaih872ddrbf1o538f3ab4dec9n02tukvmp9sgtil5/family1/bbc5f0f408a2451eb73bae2019e6c9b9 into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_) 2023-02-09 14:55:26,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__f1qtrk1rjas7bcr7lluc0gaaih872ddrbf1o538f3ab4dec9n02tukvmp9sgtil5/family1/bbc5f0f408a2451eb73bae2019e6c9b9 2023-02-09 14:55:26,628 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:26,629 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:26,630 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 2023-02-09 14:55:26,631 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e 2023-02-09 14:55:26,632 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd 2023-02-09 14:55:26,633 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 2023-02-09 14:55:26,633 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:26,633 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 14:55:26,633 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:27,115 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-09 14:55:27,115 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:27,115 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x523b2ecf to 127.0.0.1:56347 2023-02-09 14:55:27,115 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:27,121 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a59512c to 127.0.0.1:56347 2023-02-09 14:55:27,121 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:27,122 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:27,122 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1de41312 to 127.0.0.1:56347 2023-02-09 14:55:27,122 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:27,125 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:27,125 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6080aca5 to 127.0.0.1:56347 2023-02-09 14:55:27,125 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:27,523 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb 2023-02-09 14:55:27,524 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:27,531 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0 2023-02-09 14:55:27,532 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:27,542 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,542 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,550 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,550 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,551 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8303be64cb6e47d6927779db2989a8df first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 14:55:27,553 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8303be64cb6e47d6927779db2989a8df}] 2023-02-09 14:55:27,555 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8303be64cb6e47d6927779db2989a8df for inclusion in 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:27,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 14:55:27,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 14:55:27,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:27,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/8303be64cb6e47d6927779db2989a8df to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__mdejsr8ervqia939si29q1e8eb1mf5i376cng4igv3bc5u5ata4d3269pd8vafii/family1/8303be64cb6e47d6927779db2989a8df 2023-02-09 14:55:27,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__mdejsr8ervqia939si29q1e8eb1mf5i376cng4igv3bc5u5ata4d3269pd8vafii/family1/8303be64cb6e47d6927779db2989a8df as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ 2023-02-09 14:55:27,561 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/336873f8fdd84fd4b045f01beb88aaf1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 14:55:27,562 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/336873f8fdd84fd4b045f01beb88aaf1}] 2023-02-09 14:55:27,562 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__mdejsr8ervqia939si29q1e8eb1mf5i376cng4igv3bc5u5ata4d3269pd8vafii/family1/8303be64cb6e47d6927779db2989a8df into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ - updating store file list. 2023-02-09 14:55:27,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/336873f8fdd84fd4b045f01beb88aaf1 for inclusion in bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:27,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:27,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 14:55:27,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 14:55:27,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__mdejsr8ervqia939si29q1e8eb1mf5i376cng4igv3bc5u5ata4d3269pd8vafii/family1/8303be64cb6e47d6927779db2989a8df into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_) 2023-02-09 14:55:27,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:27,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__mdejsr8ervqia939si29q1e8eb1mf5i376cng4igv3bc5u5ata4d3269pd8vafii/family1/8303be64cb6e47d6927779db2989a8df 2023-02-09 14:55:27,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/336873f8fdd84fd4b045f01beb88aaf1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__65bmhme0699v1h8e655eifq062mgoep72a8q2qf21srtfdds9p1fgb5hba9oqr3o/family1/336873f8fdd84fd4b045f01beb88aaf1 2023-02-09 14:55:27,569 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:27,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__65bmhme0699v1h8e655eifq062mgoep72a8q2qf21srtfdds9p1fgb5hba9oqr3o/family1/336873f8fdd84fd4b045f01beb88aaf1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ 2023-02-09 14:55:27,569 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:27,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__65bmhme0699v1h8e655eifq062mgoep72a8q2qf21srtfdds9p1fgb5hba9oqr3o/family1/336873f8fdd84fd4b045f01beb88aaf1 into bb3d06ff69b5e78251fb182a52cc2b7b/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ - updating store file list. 2023-02-09 14:55:27,570 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 2023-02-09 14:55:27,571 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 2023-02-09 14:55:27,572 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae 2023-02-09 14:55:27,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ into bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:27,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__65bmhme0699v1h8e655eifq062mgoep72a8q2qf21srtfdds9p1fgb5hba9oqr3o/family1/336873f8fdd84fd4b045f01beb88aaf1 into bb3d06ff69b5e78251fb182a52cc2b7b/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_) 2023-02-09 14:55:27,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__65bmhme0699v1h8e655eifq062mgoep72a8q2qf21srtfdds9p1fgb5hba9oqr3o/family1/336873f8fdd84fd4b045f01beb88aaf1 2023-02-09 14:55:27,573 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a 2023-02-09 14:55:27,573 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:27,573 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 14:55:27,574 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:27,575 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:27,575 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:27,577 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 2023-02-09 14:55:27,578 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee 2023-02-09 14:55:27,579 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 2023-02-09 14:55:27,580 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 2023-02-09 14:55:27,581 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:27,581 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 14:55:27,581 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:27,581 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:27,581 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:27,581 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del] 2023-02-09 14:55:27,582 DEBUG [Listener at localhost/33189] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:27,583 DEBUG [Listener at localhost/33189] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302093ab5822404524041a87c839538268508_del 2023-02-09 14:55:27,583 INFO [Listener at localhost/33189] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 14:55:27,583 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del] 2023-02-09 14:55:27,584 DEBUG [Listener at localhost/33189] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:27,585 DEBUG [Listener at localhost/33189] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097a05711ceb154dac88e122067f05ebf3_del 2023-02-09 14:55:27,585 INFO [Listener at localhost/33189] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 14:55:27,585 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del] 2023-02-09 14:55:27,585 DEBUG [Listener at localhost/33189] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:27,587 DEBUG [Listener at localhost/33189] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209edec6ad411bd4dab9fb4e28df6bf3d8c_del 2023-02-09 14:55:27,587 INFO [Listener at localhost/33189] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 14:55:27,590 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:27,592 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:27,595 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:27,596 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:27,600 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,600 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,601 INFO [Listener at localhost/33189] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-09 14:55:27,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-09 14:55:27,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-09 14:55:27,603 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,603 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:27,604 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954527604"}]},"ts":"1675954527604"} 2023-02-09 14:55:27,605 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-09 14:55:27,607 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-09 14:55:27,607 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, UNASSIGN}] 2023-02-09 14:55:27,609 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, UNASSIGN 2023-02-09 14:55:27,609 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, UNASSIGN 2023-02-09 14:55:27,609 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, UNASSIGN 2023-02-09 14:55:27,609 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:27,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:27,609 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:27,610 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954527609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954527609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954527609"}]},"ts":"1675954527609"} 2023-02-09 14:55:27,610 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954527609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954527609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954527609"}]},"ts":"1675954527609"} 2023-02-09 14:55:27,610 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954527609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954527609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954527609"}]},"ts":"1675954527609"} 2023-02-09 14:55:27,611 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:27,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:27,612 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:27,704 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,704 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:27,720 ERROR [ProcedureDispatcherTimeoutThread] procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread(317): DelayQueue for RemoteProcedureDispatcher is not empty when timed waiting elapsed. If this is repeated consistently, it means no element is getting expired from the queue and it might freeze the system. Queue: [containedObject=jenkins-hbase4.apache.org,43455,1675954416872, timeout=1675954527762, delay=42, operations=[pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872, pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872, pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872]] 2023-02-09 14:55:27,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:27,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0f4e5cd8c7b1d17e86c4691d686f3baf, disabling compactions & flushes 2023-02-09 14:55:27,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:27,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:27,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. after waiting 0 ms 2023-02-09 14:55:27,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:27,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 14:55:27,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:27,768 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: Waiting for close lock at 1675954527763Running coprocessor pre-close hooks at 1675954527763Disabling compacts and flushes for region at 1675954527763Disabling writes for close at 1675954527763Writing region close event to WAL at 1675954527764 (+1 ms)Running coprocessor post-close hooks at 1675954527767 (+3 ms)Closed at 1675954527768 (+1 ms) 2023-02-09 14:55:27,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:27,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:27,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bb3d06ff69b5e78251fb182a52cc2b7b, disabling compactions & flushes 2023-02-09 14:55:27,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:27,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:27,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. after waiting 0 ms 2023-02-09 14:55:27,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:27,770 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=CLOSED 2023-02-09 14:55:27,770 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954527770"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954527770"}]},"ts":"1675954527770"} 2023-02-09 14:55:27,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-02-09 14:55:27,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872 in 160 msec 2023-02-09 14:55:27,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 14:55:27,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, UNASSIGN in 165 msec 2023-02-09 14:55:27,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:27,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bb3d06ff69b5e78251fb182a52cc2b7b: Waiting for close lock at 1675954527769Running coprocessor pre-close hooks at 1675954527769Disabling compacts and flushes for region at 1675954527769Disabling writes for close at 1675954527770 (+1 ms)Writing region close event to WAL at 1675954527771 (+1 ms)Running coprocessor post-close hooks at 1675954527774 (+3 ms)Closed at 1675954527774 2023-02-09 14:55:27,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:27,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:27,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 45873f0ac0c88ebf31d0fcc61f3d2515, disabling compactions & flushes 2023-02-09 14:55:27,776 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:27,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:27,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. after waiting 0 ms 2023-02-09 14:55:27,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:27,776 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=CLOSED 2023-02-09 14:55:27,776 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954527776"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954527776"}]},"ts":"1675954527776"} 2023-02-09 14:55:27,778 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-02-09 14:55:27,778 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872 in 166 msec 2023-02-09 14:55:27,779 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, UNASSIGN in 171 msec 2023-02-09 14:55:27,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 14:55:27,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:27,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 45873f0ac0c88ebf31d0fcc61f3d2515: Waiting for close lock at 1675954527776Running coprocessor pre-close hooks at 1675954527776Disabling compacts and flushes for region at 1675954527776Disabling writes for close at 1675954527776Writing region close event to WAL at 1675954527777 (+1 ms)Running coprocessor post-close hooks at 1675954527780 (+3 ms)Closed at 1675954527780 2023-02-09 14:55:27,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:27,781 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=CLOSED 2023-02-09 14:55:27,782 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954527781"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954527781"}]},"ts":"1675954527781"} 2023-02-09 14:55:27,783 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-02-09 14:55:27,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872 in 170 msec 2023-02-09 14:55:27,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=88 2023-02-09 14:55:27,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, UNASSIGN in 177 msec 2023-02-09 14:55:27,785 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954527785"}]},"ts":"1675954527785"} 2023-02-09 14:55:27,786 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-09 14:55:27,787 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-09 14:55:27,788 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 187 msec 2023-02-09 14:55:27,905 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,906 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:27,906 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-09 14:55:27,907 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,907 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,917 INFO [Listener at localhost/33189] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:27,917 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:27,917 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:27,917 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:27,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675954527918 (current time:1675954527918). 2023-02-09 14:55:27,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 14:55:27,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] zookeeper.ReadOnlyZKClient(139): Connect 0x3f6f3877 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:27,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@301e930c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:27,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:27,925 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46858, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:27,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f6f3877 to 127.0.0.1:56347 2023-02-09 14:55:27,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:27,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 14:55:27,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:27,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:27,929 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:27,929 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675955127929 2023-02-09 14:55:27,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:27,929 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1675954515428-1675954527917 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 14:55:27,930 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:27,930 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 14:55:28,030 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:28,030 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:28,030 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:28,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:28,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:28,031 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 14:55:28,231 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:28,231 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:28,231 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:28,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:28,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:28,232 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 14:55:28,336 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 14:55:28,338 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/7ca41bb69b6b4c519213ae5bac615839, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_] hfiles 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4378510017b0419aba2abe9bc0f93f51, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_] hfiles 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/e187b6bd1f0546eda9ca224a94a7ae26] hfiles 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4378510017b0419aba2abe9bc0f93f51 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/7ca41bb69b6b4c519213ae5bac615839 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb] hfiles 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/e187b6bd1f0546eda9ca224a94a7ae26 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb 2023-02-09 14:55:28,341 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/d4f20b605f20487789e4112a5736a023] hfiles 2023-02-09 14:55:28,342 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6d2440cb877e465fa091f747724714ee] hfiles 2023-02-09 14:55:28,342 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/d4f20b605f20487789e4112a5736a023 2023-02-09 14:55:28,342 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6d2440cb877e465fa091f747724714ee 2023-02-09 14:55:28,342 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/84135e5cb2f2433a95e61adda7d9b8da] hfiles 2023-02-09 14:55:28,342 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/84135e5cb2f2433a95e61adda7d9b8da 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53] hfiles 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020906fb2d7a05ca459896694c76f9a0795b 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020940984b52efb8439d84cbb982241d75f1 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209514d7ba2a7f142d9b583651d12b32d76 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209d0a90bbb19cc4705a422420b0b2a6777 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302090808519f44974110b8ea5631af3b8d4d 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302094396782798504ccf852617de118db51e 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209d979d9d0d8f542c8a094c3c168aa5119 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209dc1e147728c24497bcce05b284deb7e7 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209185f3bea5f5a4431a28e6c1518ac5106 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209c8b736a487d54797b5205cfd99b0d4c3 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209cfc7fa1699ea4004a5601e979ece577c 2023-02-09 14:55:28,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209eceda475d82246b98b4040e7b2fece53 2023-02-09 14:55:28,532 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:28,532 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:28,533 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:28,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:28,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:28,533 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 14:55:28,767 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-09 14:55:28,767 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:28,767 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:29,034 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:29,034 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:29,034 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:29,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:29,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 14:55:29,035 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 14:55:29,189 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675954515428-1675954527917 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:30,003 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1675954515428-1675954527917 of table testCompactionWithHFileLink completed 2023-02-09 14:55:30,003 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:30,004 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:30,004 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1675954515428-1675954527917 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675954527929Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1675954528336 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1675954515428-1675954527917 at 1675954528767 (+431 ms)Loading Region manifests for hbase-failsafe-snaptb-1675954515428-1675954527917 at 1675954528767Writing data manifest for hbase-failsafe-snaptb-1675954515428-1675954527917 at 1675954528770 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1675954515428-1675954527917 at 1675954529176 (+406 ms)Snapshot hbase-failsafe-snaptb-1675954515428-1675954527917 of table testCompactionWithHFileLink completed at 1675954530003 (+827 ms) 2023-02-09 14:55:30,005 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 14:55:30,006 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0770 sec 2023-02-09 14:55:30,035 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 14:55:30,035 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,035 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 14:55:30,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1675954515428-1675954527917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 14:55:30,037 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,037 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675954515428" table: "testCompactionWithHFileLink" creation_time: 1675954515438 type: FLUSH version: 2 ttl: 0 ) 2023-02-09 14:55:30,076 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1675954515428 as table=testCompactionWithHFileLink 2023-02-09 14:55:30,078 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,078 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:30,179 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,179 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:30,380 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,380 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:30,485 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 14:55:30,486 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:30,486 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 14:55:30,487 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 14:55:30,492 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1675954515428" table: "testCompactionWithHFileLink" creation_time: 1675954515438 type: FLUSH version: 2 ttl: 0 2023-02-09 14:55:30,493 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink 2023-02-09 14:55:30,497 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-09 14:55:30,500 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020909c7260163354235bffea2be1e11b8c0 2023-02-09 14:55:30,501 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209326120b51d894691984295bf07d5bfdb 2023-02-09 14:55:30,502 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302096bfa448389194bfc9d572452063fe913 2023-02-09 14:55:30,502 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,505 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,508 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,511 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,514 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,517 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,520 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,522 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,525 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,528 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,530 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,533 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a of snapshot=snaptb-1675954515428 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 14:55:30,537 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209a8b490ba7396497e914d93afe3e69a9d_del 2023-02-09 14:55:30,538 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209955e210c39604f4aa6d286cb3cb6fa1a_del 2023-02-09 14:55:30,539 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209c30afb40ef0c4eebbf49180271440e54_del 2023-02-09 14:55:30,539 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:30,539 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:30,539 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:30,542 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/e187b6bd1f0546eda9ca224a94a7ae26 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/e187b6bd1f0546eda9ca224a94a7ae26 2023-02-09 14:55:30,543 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/7ca41bb69b6b4c519213ae5bac615839 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/7ca41bb69b6b4c519213ae5bac615839 2023-02-09 14:55:30,543 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/ede91138bbb64612bd5de07f8b5a4870_SeqId_29_ 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/8d0923fb69b344628105cadf88e6d325_SeqId_29_ 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink af7831ca485342db8c538339e234fdc4 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8f34863f42864b38848944e47e9af186_SeqId_29_ 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ecd1a90c9bce4d668e4db5c09795f947 of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4378510017b0419aba2abe9bc0f93f51 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4378510017b0419aba2abe9bc0f93f51 2023-02-09 14:55:30,544 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c737342c58ad431c8113d091ddee0a2b of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,554 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 362a4118a0014b19a7654a57ad370173 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,555 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f13bcf5f89e9491db5b90e31e0bd5aa8 of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,555 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cd8327f927a7450a895525be078bf12a of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,560 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 36650b407d0d483594a88d050743a70d of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,561 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 53e3f1923aa043b4b60ae7ff7486d95a of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,562 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3ee6ce1303934fb0934193cf812d8f92 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,568 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c33cf885662b4628a3cb7f31236e384a of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,572 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 464954a5d01a4933b2b28fc2837c42ab of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,574 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink de20c5a9e953467cafa10278e6044c1e of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,576 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/84135e5cb2f2433a95e61adda7d9b8da to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/84135e5cb2f2433a95e61adda7d9b8da 2023-02-09 14:55:30,576 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fed4b332e11d4aebb3b1feb4ac19aa74 of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,578 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/d4f20b605f20487789e4112a5736a023 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/d4f20b605f20487789e4112a5736a023 2023-02-09 14:55:30,578 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 389bf55f1d4f48b3a3abe466d8e22257 of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,580 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6d2440cb877e465fa091f747724714ee to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/6d2440cb877e465fa091f747724714ee 2023-02-09 14:55:30,580 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d3eef8aad70b4a0696d86297ddf49f38 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,580 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ec14e07f0451485ca28fb2f19704e7fd of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,581 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f64b69aaaf5448ebab3776de43936cf4 of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,583 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5cf9cb87440b48f1acd34d12d8cf054d of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,583 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 515a4a199e4b4a088edbdb32e2cc9ad9 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,585 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 189968c135f743b2818d742b290ad7e4 of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,586 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b7bf9bdd44174d6aa109091b2cad7fe1 of snapshot=snaptb-1675954515428 to region=0f4e5cd8c7b1d17e86c4691d686f3baf table=testCompactionWithHFileLink 2023-02-09 14:55:30,587 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bfd9d4fc5ead40caab8aa9d276ac1773 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,588 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 566bbbea376e4418a50116016187e529 of snapshot=snaptb-1675954515428 to region=bb3d06ff69b5e78251fb182a52cc2b7b table=testCompactionWithHFileLink 2023-02-09 14:55:30,590 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a426dfd82b5a434ea24052a45c394de0 of snapshot=snaptb-1675954515428 to region=45873f0ac0c88ebf31d0fcc61f3d2515 table=testCompactionWithHFileLink 2023-02-09 14:55:30,593 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1675954515428" table: "testCompactionWithHFileLink" creation_time: 1675954515438 type: FLUSH version: 2 ttl: 0 2023-02-09 14:55:30,594 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954530594"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:30,594 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954530594"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:30,594 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954530594"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:30,596 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:55:30,596 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0f4e5cd8c7b1d17e86c4691d686f3baf, NAME => 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 45873f0ac0c88ebf31d0fcc61f3d2515, NAME => 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => bb3d06ff69b5e78251fb182a52cc2b7b, NAME => 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 14:55:30,596 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954530595"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954530595"}]},"ts":"1675954530595"} 2023-02-09 14:55:30,596 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954530595"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954530595"}]},"ts":"1675954530595"} 2023-02-09 14:55:30,596 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954530595"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954530595"}]},"ts":"1675954530595"} 2023-02-09 14:55:30,597 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 14:55:30,597 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-09 14:55:30,598 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 0f4e5cd8c7b1d17e86c4691d686f3baf, NAME => 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 45873f0ac0c88ebf31d0fcc61f3d2515, NAME => 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => bb3d06ff69b5e78251fb182a52cc2b7b, NAME => 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 14:55:30,598 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1675954515428 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-09 14:55:30,600 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675954515428" table: "testCompactionWithHFileLink" creation_time: 1675954515438 type: FLUSH version: 2 ttl: 0 ) in 532 msec 2023-02-09 14:55:30,681 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,682 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:30,683 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-09 14:55:30,683 INFO [Listener at localhost/33189] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:30,683 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,683 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1675954515428-1675954527917" 2023-02-09 14:55:30,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1675954515428-1675954527917 2023-02-09 14:55:30,691 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,691 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,691 INFO [Listener at localhost/33189] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-09 14:55:30,695 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-09 14:55:30,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-09 14:55:30,703 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,703 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,703 INFO [PEWorker-1] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-09 14:55:30,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954530703"}]},"ts":"1675954530703"} 2023-02-09 14:55:30,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 14:55:30,704 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 14:55:30,706 DEBUG [PEWorker-1] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-09 14:55:30,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN}] 2023-02-09 14:55:30,708 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN 2023-02-09 14:55:30,708 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN 2023-02-09 14:55:30,708 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN 2023-02-09 14:55:30,708 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 14:55:30,708 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 14:55:30,708 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 14:55:30,804 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:30,804 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:30,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 14:55:30,860 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:30,860 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:30,860 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:30,860 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954530860"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954530860"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954530860"}]},"ts":"1675954530860"} 2023-02-09 14:55:30,860 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954530860"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954530860"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954530860"}]},"ts":"1675954530860"} 2023-02-09 14:55:30,860 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954530860"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954530860"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954530860"}]},"ts":"1675954530860"} 2023-02-09 14:55:30,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:30,862 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:30,863 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=100, state=RUNNABLE; OpenRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:31,005 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:31,005 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:31,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 14:55:31,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:31,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bb3d06ff69b5e78251fb182a52cc2b7b, NAME => 'testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 14:55:31,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:31,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,017 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,018 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:31,018 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:31,019 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb3d06ff69b5e78251fb182a52cc2b7b columnFamilyName family1 2023-02-09 14:55:31,030 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab 2023-02-09 14:55:31,034 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a 2023-02-09 14:55:31,039 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b 2023-02-09 14:55:31,043 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a 2023-02-09 14:55:31,043 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(310): Store=bb3d06ff69b5e78251fb182a52cc2b7b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,043 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,044 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2 2023-02-09 14:55:31,044 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2 2023-02-09 14:55:31,045 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb3d06ff69b5e78251fb182a52cc2b7b columnFamilyName family2 2023-02-09 14:55:31,051 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4 2023-02-09 14:55:31,055 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257 2023-02-09 14:55:31,060 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529 2023-02-09 14:55:31,064 DEBUG [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4 2023-02-09 14:55:31,064 INFO [StoreOpener-bb3d06ff69b5e78251fb182a52cc2b7b-1] regionserver.HStore(310): Store=bb3d06ff69b5e78251fb182a52cc2b7b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:31,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:55:31,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bb3d06ff69b5e78251fb182a52cc2b7b; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62161059, jitterRate=-0.0737280398607254}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:31,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bb3d06ff69b5e78251fb182a52cc2b7b: Running coprocessor pre-open hook at 1675954531016Writing region info on filesystem at 1675954531016Initializing all the Stores at 1675954531017 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531017Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531017Cleaning up temporary data from old regions at 1675954531066 (+49 ms)Cleaning up detritus from prior splits at 1675954531067 (+1 ms)Running coprocessor post-open hooks at 1675954531069 (+2 ms)Region opened successfully at 1675954531069 2023-02-09 14:55:31,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., pid=101, masterSystemTime=1675954531013 2023-02-09 14:55:31,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:31,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:31,070 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:55:31,070 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:55:31,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:31,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:31,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:31,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f4e5cd8c7b1d17e86c4691d686f3baf, NAME => 'testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 14:55:31,073 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=bb3d06ff69b5e78251fb182a52cc2b7b, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:31,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,073 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954531072"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954531072"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954531072"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954531072"}]},"ts":"1675954531072"} 2023-02-09 14:55:31,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:31,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,073 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 14:55:31,073 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1912): bb3d06ff69b5e78251fb182a52cc2b7b/family1 is initiating minor compaction (all files) 2023-02-09 14:55:31,073 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bb3d06ff69b5e78251fb182a52cc2b7b/family1 in testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:31,074 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp, totalSize=21.6 K 2023-02-09 14:55:31,074 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 14:55:31,074 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1912): bb3d06ff69b5e78251fb182a52cc2b7b/family2 is initiating minor compaction (all files) 2023-02-09 14:55:31,074 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bb3d06ff69b5e78251fb182a52cc2b7b/family2 in testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:55:31,074 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp, totalSize=20.4 K 2023-02-09 14:55:31,074 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508688 2023-02-09 14:55:31,075 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,075 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508688 2023-02-09 14:55:31,075 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:31,077 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:31,077 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:31,077 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:31,077 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:31,078 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:31,078 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f4e5cd8c7b1d17e86c4691d686f3baf columnFamilyName family1 2023-02-09 14:55:31,078 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:31,079 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-02-09 14:55:31,079 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure bb3d06ff69b5e78251fb182a52cc2b7b, server=jenkins-hbase4.apache.org,43455,1675954416872 in 213 msec 2023-02-09 14:55:31,079 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:31,082 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bb3d06ff69b5e78251fb182a52cc2b7b, ASSIGN in 372 msec 2023-02-09 14:55:31,092 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d 2023-02-09 14:55:31,105 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a 2023-02-09 14:55:31,115 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947 2023-02-09 14:55:31,122 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb3d06ff69b5e78251fb182a52cc2b7b#family1#compaction#113 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:31,128 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] throttle.PressureAwareThroughputController(145): bb3d06ff69b5e78251fb182a52cc2b7b#family2#compaction#114 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:31,132 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8 2023-02-09 14:55:31,136 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(310): Store=0f4e5cd8c7b1d17e86c4691d686f3baf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,136 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,137 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2 2023-02-09 14:55:31,137 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2 2023-02-09 14:55:31,138 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f4e5cd8c7b1d17e86c4691d686f3baf columnFamilyName family2 2023-02-09 14:55:31,146 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d 2023-02-09 14:55:31,150 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1 2023-02-09 14:55:31,155 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd 2023-02-09 14:55:31,159 DEBUG [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74 2023-02-09 14:55:31,159 INFO [StoreOpener-0f4e5cd8c7b1d17e86c4691d686f3baf-1] regionserver.HStore(310): Store=0f4e5cd8c7b1d17e86c4691d686f3baf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:31,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:55:31,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0f4e5cd8c7b1d17e86c4691d686f3baf; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64871724, jitterRate=-0.033335983753204346}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:31,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: Running coprocessor pre-open hook at 1675954531073Writing region info on filesystem at 1675954531073Initializing all the Stores at 1675954531075 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531075Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531075Cleaning up temporary data from old regions at 1675954531161 (+86 ms)Cleaning up detritus from prior splits at 1675954531161Running coprocessor post-open hooks at 1675954531163 (+2 ms)Region opened successfully at 1675954531163 2023-02-09 14:55:31,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., pid=102, masterSystemTime=1675954531013 2023-02-09 14:55:31,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:31,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 14:55:31,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:31,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:31,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:31,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 45873f0ac0c88ebf31d0fcc61f3d2515, NAME => 'testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 14:55:31,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,166 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=0f4e5cd8c7b1d17e86c4691d686f3baf, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:31,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:31,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954531166"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954531166"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954531166"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954531166"}]},"ts":"1675954531166"} 2023-02-09 14:55:31,167 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,168 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:31,168 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:31,169 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45873f0ac0c88ebf31d0fcc61f3d2515 columnFamilyName family1 2023-02-09 14:55:31,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-02-09 14:55:31,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 0f4e5cd8c7b1d17e86c4691d686f3baf, server=jenkins-hbase4.apache.org,43455,1675954416872 in 305 msec 2023-02-09 14:55:31,170 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=0f4e5cd8c7b1d17e86c4691d686f3baf, ASSIGN in 462 msec 2023-02-09 14:55:31,176 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173 2023-02-09 14:55:31,181 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92 2023-02-09 14:55:31,187 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4 2023-02-09 14:55:31,195 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e 2023-02-09 14:55:31,196 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(310): Store=45873f0ac0c88ebf31d0fcc61f3d2515/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,196 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,197 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2 2023-02-09 14:55:31,197 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2 2023-02-09 14:55:31,197 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45873f0ac0c88ebf31d0fcc61f3d2515 columnFamilyName family2 2023-02-09 14:55:31,205 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9 2023-02-09 14:55:31,211 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0 2023-02-09 14:55:31,217 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773 2023-02-09 14:55:31,223 DEBUG [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(539): loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38 2023-02-09 14:55:31,223 INFO [StoreOpener-45873f0ac0c88ebf31d0fcc61f3d2515-1] regionserver.HStore(310): Store=45873f0ac0c88ebf31d0fcc61f3d2515/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:31,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:31,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:55:31,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 45873f0ac0c88ebf31d0fcc61f3d2515; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59969009, jitterRate=-0.10639213025569916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:31,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 45873f0ac0c88ebf31d0fcc61f3d2515: Running coprocessor pre-open hook at 1675954531166Writing region info on filesystem at 1675954531166Initializing all the Stores at 1675954531167 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531167Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954531167Cleaning up temporary data from old regions at 1675954531226 (+59 ms)Cleaning up detritus from prior splits at 1675954531226Running coprocessor post-open hooks at 1675954531228 (+2 ms)Region opened successfully at 1675954531228 2023-02-09 14:55:31,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., pid=103, masterSystemTime=1675954531013 2023-02-09 14:55:31,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 14:55:31,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 14:55:31,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:31,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:31,231 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=45873f0ac0c88ebf31d0fcc61f3d2515, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:31,231 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675954531231"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954531231"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954531231"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954531231"}]},"ts":"1675954531231"} 2023-02-09 14:55:31,234 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=100 2023-02-09 14:55:31,234 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=100, state=SUCCESS; OpenRegionProcedure 45873f0ac0c88ebf31d0fcc61f3d2515, server=jenkins-hbase4.apache.org,43455,1675954416872 in 369 msec 2023-02-09 14:55:31,235 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-09 14:55:31,235 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=45873f0ac0c88ebf31d0fcc61f3d2515, ASSIGN in 527 msec 2023-02-09 14:55:31,235 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954531235"}]},"ts":"1675954531235"} 2023-02-09 14:55:31,236 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 14:55:31,238 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-09 14:55:31,239 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 542 msec 2023-02-09 14:55:31,306 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:31,307 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:31,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 14:55:31,307 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-09 14:55:31,320 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:31,328 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:31,335 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:31,336 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:31,342 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:55:31,355 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 14:55:31,355 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:31,355 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(139): Connect 0x0960de82 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:31,360 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74e615de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:31,360 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:31,360 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:31,360 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:31,948 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family1/93914975adac48c4a5a898951583443b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/93914975adac48c4a5a898951583443b 2023-02-09 14:55:31,952 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bb3d06ff69b5e78251fb182a52cc2b7b/family1 of bb3d06ff69b5e78251fb182a52cc2b7b into 93914975adac48c4a5a898951583443b(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:31,952 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:31,952 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., storeName=bb3d06ff69b5e78251fb182a52cc2b7b/family1, priority=12, startTime=1675954531070; duration=0sec 2023-02-09 14:55:31,952 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 14:55:31,953 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:55:31,954 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 14:55:31,955 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1912): 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 is initiating minor compaction (all files) 2023-02-09 14:55:31,955 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 in testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:31,955 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp, totalSize=21.6 K 2023-02-09 14:55:31,955 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508687 2023-02-09 14:55:31,956 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:31,956 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512262 2023-02-09 14:55:31,956 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:31,967 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0f4e5cd8c7b1d17e86c4691d686f3baf#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:32,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab] to archive 2023-02-09 14:55:32,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:32,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-c737342c58ad431c8113d091ddee0a2b 2023-02-09 14:55:32,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-cd8327f927a7450a895525be078bf12a 2023-02-09 14:55:32,025 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-53e3f1923aa043b4b60ae7ff7486d95a 2023-02-09 14:55:32,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-464954a5d01a4933b2b28fc2837c42ab 2023-02-09 14:55:32,262 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209445514605c0348cb89c2f6d3fe37088b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209445514605c0348cb89c2f6d3fe37088b 2023-02-09 14:55:32,262 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302097e29ac2b55524a0fa9fe0f91c3a22a13 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097e29ac2b55524a0fa9fe0f91c3a22a13 2023-02-09 14:55:32,262 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302097f1acb2510f048c1abe33d83eb01ea5a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097f1acb2510f048c1abe33d83eb01ea5a 2023-02-09 14:55:32,263 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,263 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,263 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,277 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:32,280 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46860, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:32,286 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,286 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,286 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,286 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,287 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:55:32,290 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36792, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:55:32,291 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,291 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,291 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,291 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,316 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/ab740b11eada4af4952f3a7619ab97e0 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-09 14:55:32,316 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/4b5c7bd953c848fba3d6b01bde388068 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-09 14:55:32,316 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/afa56a5a3072409fbdf78746549c8bb6 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-09 14:55:32,317 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/ab740b11eada4af4952f3a7619ab97e0}] 2023-02-09 14:55:32,317 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/4b5c7bd953c848fba3d6b01bde388068}] 2023-02-09 14:55:32,317 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=32 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/afa56a5a3072409fbdf78746549c8bb6}] 2023-02-09 14:55:32,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/afa56a5a3072409fbdf78746549c8bb6 for inclusion in 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:32,326 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/4b5c7bd953c848fba3d6b01bde388068 for inclusion in bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:32,326 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/ab740b11eada4af4952f3a7619ab97e0 for inclusion in 45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 14:55:32,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 14:55:32,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:32,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:32,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:32,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/afa56a5a3072409fbdf78746549c8bb6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__cpj4jmj4jkdlvv6vf45gcbt8je2ue5mdp9ur1k75r25ujivkqa0lr6cbfin7hcur/family1/afa56a5a3072409fbdf78746549c8bb6 2023-02-09 14:55:32,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/ab740b11eada4af4952f3a7619ab97e0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__jumpdcovhikonmvd8vm0hssu864ha3f1flqj368ba0veggpa7j5l8m9m14i1q5ee/family1/ab740b11eada4af4952f3a7619ab97e0 2023-02-09 14:55:32,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/4b5c7bd953c848fba3d6b01bde388068 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__b8e7noanspqb2u0v3e63au4ugd1m7i5up2k1f0crj5271tg6hre006ql526g0fe1/family1/4b5c7bd953c848fba3d6b01bde388068 2023-02-09 14:55:32,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__cpj4jmj4jkdlvv6vf45gcbt8je2ue5mdp9ur1k75r25ujivkqa0lr6cbfin7hcur/family1/afa56a5a3072409fbdf78746549c8bb6 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8870bcb2c8404917a37e12952bf01d35_SeqId_34_ 2023-02-09 14:55:32,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__jumpdcovhikonmvd8vm0hssu864ha3f1flqj368ba0veggpa7j5l8m9m14i1q5ee/family1/ab740b11eada4af4952f3a7619ab97e0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_ 2023-02-09 14:55:32,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__b8e7noanspqb2u0v3e63au4ugd1m7i5up2k1f0crj5271tg6hre006ql526g0fe1/family1/4b5c7bd953c848fba3d6b01bde388068 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4584bafce5b74312a6a53713a5e27100_SeqId_35_ 2023-02-09 14:55:32,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__cpj4jmj4jkdlvv6vf45gcbt8je2ue5mdp9ur1k75r25ujivkqa0lr6cbfin7hcur/family1/afa56a5a3072409fbdf78746549c8bb6 into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8870bcb2c8404917a37e12952bf01d35_SeqId_34_ - updating store file list. 2023-02-09 14:55:32,334 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__jumpdcovhikonmvd8vm0hssu864ha3f1flqj368ba0veggpa7j5l8m9m14i1q5ee/family1/ab740b11eada4af4952f3a7619ab97e0 into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_ - updating store file list. 2023-02-09 14:55:32,335 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__b8e7noanspqb2u0v3e63au4ugd1m7i5up2k1f0crj5271tg6hre006ql526g0fe1/family1/4b5c7bd953c848fba3d6b01bde388068 into bb3d06ff69b5e78251fb182a52cc2b7b/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4584bafce5b74312a6a53713a5e27100_SeqId_35_ - updating store file list. 2023-02-09 14:55:32,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_ into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 2023-02-09 14:55:32,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__jumpdcovhikonmvd8vm0hssu864ha3f1flqj368ba0veggpa7j5l8m9m14i1q5ee/family1/ab740b11eada4af4952f3a7619ab97e0 into 45873f0ac0c88ebf31d0fcc61f3d2515/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_) 2023-02-09 14:55:32,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__jumpdcovhikonmvd8vm0hssu864ha3f1flqj368ba0veggpa7j5l8m9m14i1q5ee/family1/ab740b11eada4af4952f3a7619ab97e0 2023-02-09 14:55:32,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4584bafce5b74312a6a53713a5e27100_SeqId_35_ into bb3d06ff69b5e78251fb182a52cc2b7b/family1 2023-02-09 14:55:32,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:32,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8870bcb2c8404917a37e12952bf01d35_SeqId_34_ into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 2023-02-09 14:55:32,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__b8e7noanspqb2u0v3e63au4ugd1m7i5up2k1f0crj5271tg6hre006ql526g0fe1/family1/4b5c7bd953c848fba3d6b01bde388068 into bb3d06ff69b5e78251fb182a52cc2b7b/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family1/4584bafce5b74312a6a53713a5e27100_SeqId_35_) 2023-02-09 14:55:32,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__cpj4jmj4jkdlvv6vf45gcbt8je2ue5mdp9ur1k75r25ujivkqa0lr6cbfin7hcur/family1/afa56a5a3072409fbdf78746549c8bb6 into 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/8870bcb2c8404917a37e12952bf01d35_SeqId_34_) 2023-02-09 14:55:32,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__b8e7noanspqb2u0v3e63au4ugd1m7i5up2k1f0crj5271tg6hre006ql526g0fe1/family1/4b5c7bd953c848fba3d6b01bde388068 2023-02-09 14:55:32,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testCompactionWithHFileLink__cpj4jmj4jkdlvv6vf45gcbt8je2ue5mdp9ur1k75r25ujivkqa0lr6cbfin7hcur/family1/afa56a5a3072409fbdf78746549c8bb6 2023-02-09 14:55:32,342 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,343 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,343 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:32,343 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:32,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-09 14:55:32,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(1912): 45873f0ac0c88ebf31d0fcc61f3d2515/family1 is initiating minor compaction (all files) 2023-02-09 14:55:32,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5a350b70; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 14:55:32,345 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ab8864c2f95741cbb1656224847bdda1 2023-02-09 14:55:32,345 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302093bf791f5136c44b28eae35dd22e48787 2023-02-09 14:55:32,346 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ae385b58513d49b2aa6d4304bc0db5d2 2023-02-09 14:55:32,347 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302098457388a4b7d4df0bcd7aed7a6df05ee 2023-02-09 14:55:32,347 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,348 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209d0d625be7b0b4419bc522015f5d87a21 2023-02-09 14:55:32,349 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:32,349 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209ce7acb5720484d24ae3be3de8e8145ae 2023-02-09 14:55:32,350 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209ed394920761942f8aa3c9e7061ec2104 2023-02-09 14:55:32,350 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:32,350 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 14:55:32,350 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209d0b5bca691704adc83f8f0892869722a 2023-02-09 14:55:32,350 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:32,350 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 14:55:32,350 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020901e6693d5e034506b6fb14843aa49d17 2023-02-09 14:55:32,350 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:32,351 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:32,352 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209049e87550cd44aa7b7adf9a91e81e39e 2023-02-09 14:55:32,352 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/.tmp/family2/46b89a84b0db4bcca8d4ec10a31cb7c3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/46b89a84b0db4bcca8d4ec10a31cb7c3 2023-02-09 14:55:32,353 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209dafc22f0d5c54a20aefe7c2d5051d1dd 2023-02-09 14:55:32,354 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209e41f67dbf146425e9f42f7551f49f3f8 2023-02-09 14:55:32,354 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 14:55:32,354 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 14:55:32,354 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 14:55:32,354 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 14:55:32,357 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bb3d06ff69b5e78251fb182a52cc2b7b/family2 of bb3d06ff69b5e78251fb182a52cc2b7b into 46b89a84b0db4bcca8d4ec10a31cb7c3(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:32,357 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb3d06ff69b5e78251fb182a52cc2b7b: 2023-02-09 14:55:32,357 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b., storeName=bb3d06ff69b5e78251fb182a52cc2b7b/family2, priority=12, startTime=1675954531070; duration=1sec 2023-02-09 14:55:32,357 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 14:55:32,357 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 45873f0ac0c88ebf31d0fcc61f3d2515/family1 in testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:32,357 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp, totalSize=28.9 K 2023-02-09 14:55:32,358 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508689 2023-02-09 14:55:32,358 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:32,358 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:32,360 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:32,360 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:32,361 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting 6e41b2163c3d487388ae5d58957c1544_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675954508689 2023-02-09 14:55:32,365 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:32,368 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:32,370 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:32,379 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:32,380 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:32,381 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] throttle.PressureAwareThroughputController(145): 45873f0ac0c88ebf31d0fcc61f3d2515#family1#compaction#116 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:32,398 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=287 (was 245) - Thread LEAK? -, OpenFileDescriptor=764 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=99 (was 139), ProcessCount=168 (was 168), AvailableMemoryMB=2984 (was 3065) 2023-02-09 14:55:32,412 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=764, MaxFileDescriptor=60000, SystemLoadAverage=99, ProcessCount=168, AvailableMemoryMB=2983 2023-02-09 14:55:32,415 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,415 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529] to archive 2023-02-09 14:55:32,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:32,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-389bf55f1d4f48b3a3abe466d8e22257 2023-02-09 14:55:32,419 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-09 14:55:32,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-f64b69aaaf5448ebab3776de43936cf4 2023-02-09 14:55:32,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-09 14:55:32,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-189968c135f743b2818d742b290ad7e4 2023-02-09 14:55:32,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/family2/testCompactionWithHFileLink=bb3d06ff69b5e78251fb182a52cc2b7b-566bbbea376e4418a50116016187e529 2023-02-09 14:55:32,425 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,425 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 14:55:32,427 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:55:32,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-02-09 14:55:32,526 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,526 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 14:55:32,527 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,527 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 14:55:32,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-09 14:55:32,531 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:32,531 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-09 14:55:32,531 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,531 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,531 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:32,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:32,533 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000 empty. 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2 empty. 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b empty. 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:32,534 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:32,535 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:32,535 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-09 14:55:32,542 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:32,543 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9a00c85257e134deaed37338dc2b5000, NAME => 'ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:32,544 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => a92002b17d4c5e771b3092e4c4816fd2, NAME => 'ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:32,544 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 003f0df30fea469fcf63c1030019683b, NAME => 'ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:32,632 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,632 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:32,834 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,834 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 9a00c85257e134deaed37338dc2b5000, disabling compactions & flushes 2023-02-09 14:55:32,952 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. after waiting 0 ms 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:32,952 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:32,952 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 9a00c85257e134deaed37338dc2b5000: Waiting for close lock at 1675954532952Disabling compacts and flushes for region at 1675954532952Disabling writes for close at 1675954532952Writing region close event to WAL at 1675954532952Closed at 1675954532952 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing a92002b17d4c5e771b3092e4c4816fd2, disabling compactions & flushes 2023-02-09 14:55:32,954 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. after waiting 0 ms 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:32,954 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:32,954 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for a92002b17d4c5e771b3092e4c4816fd2: Waiting for close lock at 1675954532954Disabling compacts and flushes for region at 1675954532954Disabling writes for close at 1675954532954Writing region close event to WAL at 1675954532954Closed at 1675954532954 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 003f0df30fea469fcf63c1030019683b, disabling compactions & flushes 2023-02-09 14:55:32,958 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. after waiting 0 ms 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:32,958 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:32,958 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 003f0df30fea469fcf63c1030019683b: Waiting for close lock at 1675954532958Disabling compacts and flushes for region at 1675954532958Disabling writes for close at 1675954532958Writing region close event to WAL at 1675954532958Closed at 1675954532958 2023-02-09 14:55:32,960 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:32,961 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954532961"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954532961"}]},"ts":"1675954532961"} 2023-02-09 14:55:32,961 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675954532961"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954532961"}]},"ts":"1675954532961"} 2023-02-09 14:55:32,961 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954532961"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954532961"}]},"ts":"1675954532961"} 2023-02-09 14:55:32,963 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 14:55:32,963 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:32,964 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954532963"}]},"ts":"1675954532963"} 2023-02-09 14:55:32,965 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-09 14:55:32,967 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a00c85257e134deaed37338dc2b5000, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a92002b17d4c5e771b3092e4c4816fd2, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=003f0df30fea469fcf63c1030019683b, ASSIGN}] 2023-02-09 14:55:32,969 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a00c85257e134deaed37338dc2b5000, ASSIGN 2023-02-09 14:55:32,969 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=003f0df30fea469fcf63c1030019683b, ASSIGN 2023-02-09 14:55:32,969 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a92002b17d4c5e771b3092e4c4816fd2, ASSIGN 2023-02-09 14:55:32,969 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a00c85257e134deaed37338dc2b5000, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:32,969 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=003f0df30fea469fcf63c1030019683b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:32,970 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a92002b17d4c5e771b3092e4c4816fd2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:33,122 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=003f0df30fea469fcf63c1030019683b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,122 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9a00c85257e134deaed37338dc2b5000, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,122 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a92002b17d4c5e771b3092e4c4816fd2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,122 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954533122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954533122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954533122"}]},"ts":"1675954533122"} 2023-02-09 14:55:33,122 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954533122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954533122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954533122"}]},"ts":"1675954533122"} 2023-02-09 14:55:33,122 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675954533122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954533122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954533122"}]},"ts":"1675954533122"} 2023-02-09 14:55:33,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 003f0df30fea469fcf63c1030019683b, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:33,124 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 9a00c85257e134deaed37338dc2b5000, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:33,124 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure a92002b17d4c5e771b3092e4c4816fd2, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:33,135 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:33,135 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:33,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:33,187 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family1/0dd94275dd0d4ef48afba4d628f5716d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/0dd94275dd0d4ef48afba4d628f5716d 2023-02-09 14:55:33,190 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 0f4e5cd8c7b1d17e86c4691d686f3baf/family1 of 0f4e5cd8c7b1d17e86c4691d686f3baf into 0dd94275dd0d4ef48afba4d628f5716d(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:33,190 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:33,190 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., storeName=0f4e5cd8c7b1d17e86c4691d686f3baf/family1, priority=12, startTime=1675954531164; duration=1sec 2023-02-09 14:55:33,190 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 14:55:33,191 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:55:33,192 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 14:55:33,192 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1912): 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 is initiating minor compaction (all files) 2023-02-09 14:55:33,192 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 in testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:55:33,193 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp, totalSize=20.4 K 2023-02-09 14:55:33,193 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508687 2023-02-09 14:55:33,194 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:33,194 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512262 2023-02-09 14:55:33,195 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:33,202 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family1/cee5771ebab54b3e8af4e0e2d4344b8a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/cee5771ebab54b3e8af4e0e2d4344b8a 2023-02-09 14:55:33,206 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 45873f0ac0c88ebf31d0fcc61f3d2515/family1 of 45873f0ac0c88ebf31d0fcc61f3d2515 into cee5771ebab54b3e8af4e0e2d4344b8a(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:33,206 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:33,206 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675954532344; duration=0sec 2023-02-09 14:55:33,206 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 14:55:33,206 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0f4e5cd8c7b1d17e86c4691d686f3baf#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:33,206 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:55:33,208 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 14:55:33,208 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1912): 45873f0ac0c88ebf31d0fcc61f3d2515/family2 is initiating minor compaction (all files) 2023-02-09 14:55:33,208 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 45873f0ac0c88ebf31d0fcc61f3d2515/family2 in testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:55:33,209 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp, totalSize=20.4 K 2023-02-09 14:55:33,209 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954508689 2023-02-09 14:55:33,210 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954510473 2023-02-09 14:55:33,210 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954512263 2023-02-09 14:55:33,211 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954514045 2023-02-09 14:55:33,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a] to archive 2023-02-09 14:55:33,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:33,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ecd1a90c9bce4d668e4db5c09795f947 2023-02-09 14:55:33,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-f13bcf5f89e9491db5b90e31e0bd5aa8 2023-02-09 14:55:33,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-36650b407d0d483594a88d050743a70d 2023-02-09 14:55:33,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family1/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-c33cf885662b4628a3cb7f31236e384a 2023-02-09 14:55:33,222 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] throttle.PressureAwareThroughputController(145): 45873f0ac0c88ebf31d0fcc61f3d2515#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 73.08 MB/second 2023-02-09 14:55:33,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_] to archive 2023-02-09 14:55:33,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:33,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-af7831ca485342db8c538339e234fdc4 2023-02-09 14:55:33,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-362a4118a0014b19a7654a57ad370173 2023-02-09 14:55:33,226 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-3ee6ce1303934fb0934193cf812d8f92 2023-02-09 14:55:33,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-de20c5a9e953467cafa10278e6044c1e 2023-02-09 14:55:33,228 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family1/6e41b2163c3d487388ae5d58957c1544_SeqId_34_ 2023-02-09 14:55:33,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:33,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a00c85257e134deaed37338dc2b5000, NAME => 'ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 14:55:33,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:33,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,279 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,280 DEBUG [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1 2023-02-09 14:55:33,280 DEBUG [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1 2023-02-09 14:55:33,280 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a00c85257e134deaed37338dc2b5000 columnFamilyName family1 2023-02-09 14:55:33,281 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] regionserver.HStore(310): Store=9a00c85257e134deaed37338dc2b5000/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,281 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,282 DEBUG [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2 2023-02-09 14:55:33,282 DEBUG [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2 2023-02-09 14:55:33,283 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a00c85257e134deaed37338dc2b5000 columnFamilyName family2 2023-02-09 14:55:33,283 INFO [StoreOpener-9a00c85257e134deaed37338dc2b5000-1] regionserver.HStore(310): Store=9a00c85257e134deaed37338dc2b5000/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:33,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:55:33,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:33,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a00c85257e134deaed37338dc2b5000; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59243972, jitterRate=-0.11719602346420288}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:33,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a00c85257e134deaed37338dc2b5000: Running coprocessor pre-open hook at 1675954533278Writing region info on filesystem at 1675954533278Initializing all the Stores at 1675954533278Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533278Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533279 (+1 ms)Cleaning up temporary data from old regions at 1675954533285 (+6 ms)Cleaning up detritus from prior splits at 1675954533285Running coprocessor post-open hooks at 1675954533288 (+3 ms)Region opened successfully at 1675954533288 2023-02-09 14:55:33,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000., pid=110, masterSystemTime=1675954533275 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:33,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:33,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 003f0df30fea469fcf63c1030019683b, NAME => 'ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,290 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9a00c85257e134deaed37338dc2b5000, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,290 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954533290"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954533290"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954533290"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954533290"}]},"ts":"1675954533290"} 2023-02-09 14:55:33,291 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,292 DEBUG [StoreOpener-003f0df30fea469fcf63c1030019683b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1 2023-02-09 14:55:33,292 DEBUG [StoreOpener-003f0df30fea469fcf63c1030019683b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1 2023-02-09 14:55:33,293 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-02-09 14:55:33,293 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 9a00c85257e134deaed37338dc2b5000, server=jenkins-hbase4.apache.org,43455,1675954416872 in 167 msec 2023-02-09 14:55:33,293 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 003f0df30fea469fcf63c1030019683b columnFamilyName family1 2023-02-09 14:55:33,293 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a00c85257e134deaed37338dc2b5000, ASSIGN in 326 msec 2023-02-09 14:55:33,293 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] regionserver.HStore(310): Store=003f0df30fea469fcf63c1030019683b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,294 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,295 DEBUG [StoreOpener-003f0df30fea469fcf63c1030019683b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2 2023-02-09 14:55:33,295 DEBUG [StoreOpener-003f0df30fea469fcf63c1030019683b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2 2023-02-09 14:55:33,295 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 003f0df30fea469fcf63c1030019683b columnFamilyName family2 2023-02-09 14:55:33,295 INFO [StoreOpener-003f0df30fea469fcf63c1030019683b-1] regionserver.HStore(310): Store=003f0df30fea469fcf63c1030019683b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:33,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 003f0df30fea469fcf63c1030019683b 2023-02-09 14:55:33,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:33,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 003f0df30fea469fcf63c1030019683b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72239127, jitterRate=0.07644687592983246}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:33,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 003f0df30fea469fcf63c1030019683b: Running coprocessor pre-open hook at 1675954533290Writing region info on filesystem at 1675954533290Initializing all the Stores at 1675954533291 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533291Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533291Cleaning up temporary data from old regions at 1675954533297 (+6 ms)Cleaning up detritus from prior splits at 1675954533297Running coprocessor post-open hooks at 1675954533300 (+3 ms)Region opened successfully at 1675954533300 2023-02-09 14:55:33,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b., pid=109, masterSystemTime=1675954533275 2023-02-09 14:55:33,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:33,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:33,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:33,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a92002b17d4c5e771b3092e4c4816fd2, NAME => 'ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 14:55:33,302 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=003f0df30fea469fcf63c1030019683b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,302 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675954533302"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954533302"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954533302"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954533302"}]},"ts":"1675954533302"} 2023-02-09 14:55:33,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:33,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,303 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,304 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-02-09 14:55:33,304 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 003f0df30fea469fcf63c1030019683b, server=jenkins-hbase4.apache.org,43455,1675954416872 in 180 msec 2023-02-09 14:55:33,304 DEBUG [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1 2023-02-09 14:55:33,304 DEBUG [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1 2023-02-09 14:55:33,304 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a92002b17d4c5e771b3092e4c4816fd2 columnFamilyName family1 2023-02-09 14:55:33,305 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=003f0df30fea469fcf63c1030019683b, ASSIGN in 337 msec 2023-02-09 14:55:33,305 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] regionserver.HStore(310): Store=a92002b17d4c5e771b3092e4c4816fd2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,305 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,306 DEBUG [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2 2023-02-09 14:55:33,306 DEBUG [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2 2023-02-09 14:55:33,307 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a92002b17d4c5e771b3092e4c4816fd2 columnFamilyName family2 2023-02-09 14:55:33,307 INFO [StoreOpener-a92002b17d4c5e771b3092e4c4816fd2-1] regionserver.HStore(310): Store=a92002b17d4c5e771b3092e4c4816fd2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:33,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:33,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:55:33,310 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:33,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:33,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a92002b17d4c5e771b3092e4c4816fd2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61507362, jitterRate=-0.08346888422966003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:33,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a92002b17d4c5e771b3092e4c4816fd2: Running coprocessor pre-open hook at 1675954533302Writing region info on filesystem at 1675954533302Initializing all the Stores at 1675954533303 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533303Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954533303Cleaning up temporary data from old regions at 1675954533309 (+6 ms)Cleaning up detritus from prior splits at 1675954533309Running coprocessor post-open hooks at 1675954533312 (+3 ms)Region opened successfully at 1675954533312 2023-02-09 14:55:33,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2., pid=111, masterSystemTime=1675954533275 2023-02-09 14:55:33,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:33,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:33,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=a92002b17d4c5e771b3092e4c4816fd2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675954533315"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954533315"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954533315"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954533315"}]},"ts":"1675954533315"} 2023-02-09 14:55:33,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-02-09 14:55:33,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure a92002b17d4c5e771b3092e4c4816fd2, server=jenkins-hbase4.apache.org,43455,1675954416872 in 192 msec 2023-02-09 14:55:33,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-02-09 14:55:33,319 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a92002b17d4c5e771b3092e4c4816fd2, ASSIGN in 350 msec 2023-02-09 14:55:33,319 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:33,320 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954533319"}]},"ts":"1675954533319"} 2023-02-09 14:55:33,321 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-09 14:55:33,324 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:33,325 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 795 msec 2023-02-09 14:55:33,636 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:33,636 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:33,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:33,637 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-09 14:55:33,642 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:33,642 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:33,643 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:33,644 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:33,644 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:33,645 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:33,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 14:55:33,646 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 14:55:33,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:33,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 14:55:33,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 14:55:33,647 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:33,647 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:33,648 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:33,648 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:33,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:33,648 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:33,648 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:33,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:33,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:33,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:33,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:33,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 14:55:33,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 14:55:33,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 14:55:33,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:33,649 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 14:55:33,653 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:33,653 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. started... 2023-02-09 14:55:33,653 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:33,653 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:33,655 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92002b17d4c5e771b3092e4c4816fd2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:33,655 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:33,655 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:33,655 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. started... 2023-02-09 14:55:33,653 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. started... 2023-02-09 14:55:33,655 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9a00c85257e134deaed37338dc2b5000 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:33,655 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 003f0df30fea469fcf63c1030019683b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:55:34,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302095ca4ef6e0c54476bb0961cced99d3e87 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302095ca4ef6e0c54476bb0961cced99d3e87 2023-02-09 14:55:34,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020982ffaf72f2cd482d89bc9edf1c203584 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982ffaf72f2cd482d89bc9edf1c203584 2023-02-09 14:55:34,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090aa1cc13521649eeb9026973d26c9a87 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302090aa1cc13521649eeb9026973d26c9a87 2023-02-09 14:55:34,423 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/.tmp/family2/e782f92cc09940cb951efb721b483f2e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/e782f92cc09940cb951efb721b483f2e 2023-02-09 14:55:34,426 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 0f4e5cd8c7b1d17e86c4691d686f3baf/family2 of 0f4e5cd8c7b1d17e86c4691d686f3baf into e782f92cc09940cb951efb721b483f2e(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:34,426 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: 2023-02-09 14:55:34,427 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf., storeName=0f4e5cd8c7b1d17e86c4691d686f3baf/family2, priority=12, startTime=1675954531164; duration=1sec 2023-02-09 14:55:34,427 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:34,427 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 14:55:34,427 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 14:55:34,427 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 14:55:34,427 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. because compaction request was cancelled 2023-02-09 14:55:34,436 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/.tmp/family2/c918a876a3c5400892d8fa442f47adbd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/c918a876a3c5400892d8fa442f47adbd 2023-02-09 14:55:34,439 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 45873f0ac0c88ebf31d0fcc61f3d2515/family2 of 45873f0ac0c88ebf31d0fcc61f3d2515 into c918a876a3c5400892d8fa442f47adbd(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:34,439 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 45873f0ac0c88ebf31d0fcc61f3d2515: 2023-02-09 14:55:34,439 INFO [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515., storeName=45873f0ac0c88ebf31d0fcc61f3d2515/family2, priority=12, startTime=1675954531229; duration=1sec 2023-02-09 14:55:34,439 DEBUG [RS:0;jenkins-hbase4:43455-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:34,479 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/d4da09cafeff4e65961ca123e78f6986 2023-02-09 14:55:34,479 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/fe4dadceb79643c8adaf250fdef021fd 2023-02-09 14:55:34,479 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/b6a28187a8e4488ba5901a0b16a60afa 2023-02-09 14:55:34,492 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209a061cf2285c44bf3b6b93a343f0d7ceb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209a061cf2285c44bf3b6b93a343f0d7ceb 2023-02-09 14:55:34,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1] to archive 2023-02-09 14:55:34,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:34,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-fed4b332e11d4aebb3b1feb4ac19aa74 2023-02-09 14:55:34,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-ec14e07f0451485ca28fb2f19704e7fd 2023-02-09 14:55:34,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-5cf9cb87440b48f1acd34d12d8cf054d 2023-02-09 14:55:34,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/family2/testCompactionWithHFileLink=0f4e5cd8c7b1d17e86c4691d686f3baf-b7bf9bdd44174d6aa109091b2cad7fe1 2023-02-09 14:55:34,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0] to archive 2023-02-09 14:55:34,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:34,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-d3eef8aad70b4a0696d86297ddf49f38 2023-02-09 14:55:34,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-515a4a199e4b4a088edbdb32e2cc9ad9 2023-02-09 14:55:34,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-bfd9d4fc5ead40caab8aa9d276ac1773 2023-02-09 14:55:34,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/family2/testCompactionWithHFileLink=45873f0ac0c88ebf31d0fcc61f3d2515-a426dfd82b5a434ea24052a45c394de0 2023-02-09 14:55:34,894 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209a454db6e79ff423e8398e64f766c752e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209a454db6e79ff423e8398e64f766c752e 2023-02-09 14:55:34,894 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209f41730e872094b71b88be1e03993e0c2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209f41730e872094b71b88be1e03993e0c2 2023-02-09 14:55:34,896 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/3fd2063476584601950204ca7ff39e34 2023-02-09 14:55:34,904 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/d4da09cafeff4e65961ca123e78f6986 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986 2023-02-09 14:55:34,904 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/3e0e533f69fb49ccadeab586ba5904db 2023-02-09 14:55:34,908 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:34,909 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/fe4dadceb79643c8adaf250fdef021fd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd 2023-02-09 14:55:34,909 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/3fd2063476584601950204ca7ff39e34 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34 2023-02-09 14:55:34,913 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:34,913 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:34,914 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a92002b17d4c5e771b3092e4c4816fd2 in 1260ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:34,914 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/3e0e533f69fb49ccadeab586ba5904db as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db 2023-02-09 14:55:34,914 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-09 14:55:34,915 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:34,915 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:34,915 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:34,918 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:34,919 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9a00c85257e134deaed37338dc2b5000 in 1264ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:34,919 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:34,919 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:34,920 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:35,299 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/56b77831d2c24f1b8f6588436048263b 2023-02-09 14:55:35,303 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/b6a28187a8e4488ba5901a0b16a60afa as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa 2023-02-09 14:55:35,307 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 14:55:35,307 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/56b77831d2c24f1b8f6588436048263b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b 2023-02-09 14:55:35,311 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:55:35,311 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 003f0df30fea469fcf63c1030019683b in 1656ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:35,311 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:35,311 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:35,311 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:35,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:35,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 14:55:35,312 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 14:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,313 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,314 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:35,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:35,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:35,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:35,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 14:55:35,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b5c9c29[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:35,315 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 14:55:35,315 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,316 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 14:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 14:55:35,316 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,316 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:35,317 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,317 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 14:55:35,317 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:35,317 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 14:55:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:35,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 14:55:35,319 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 14:55:35,319 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:35,319 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:35,319 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 14:55:35,319 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:35,322 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,322 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:35,322 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:35,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,323 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:35,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:35,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:35,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:35,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,324 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:35,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:35,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:35,326 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:35,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:35,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:35,326 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,326 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 14:55:35,326 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:35,326 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:35,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:35,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:35,327 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,327 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:35,327 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:35,427 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:35,427 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:35,427 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:35,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:35,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:35,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:35,429 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:35,431 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:35,431 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:35,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:35,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:35,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 14:55:35,433 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 14:55:35,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:35,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 14:55:35,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 14:55:35,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,433 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:35,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,435 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:35,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:35,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:35,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:35,435 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:35,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:35,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:35,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:35,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:35,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 14:55:35,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 14:55:35,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 14:55:35,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 14:55:35,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:35,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 14:55:35,436 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:35,436 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. started... 2023-02-09 14:55:35,436 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:35,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:35,436 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:35,436 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92002b17d4c5e771b3092e4c4816fd2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:35,436 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:35,437 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. started... 2023-02-09 14:55:35,436 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. started... 2023-02-09 14:55:35,437 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 003f0df30fea469fcf63c1030019683b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:35,437 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9a00c85257e134deaed37338dc2b5000 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:55:35,849 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d1136b9c6e8c48e9b62778af29b7419b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209d1136b9c6e8c48e9b62778af29b7419b 2023-02-09 14:55:35,849 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209e8a545c2037d491fa6e46addb297946d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209e8a545c2037d491fa6e46addb297946d 2023-02-09 14:55:35,849 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302091ab01a08762148dc9906a3cc72803bf5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302091ab01a08762148dc9906a3cc72803bf5 2023-02-09 14:55:36,254 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/20fdafcb084e4708952478593c937eaa 2023-02-09 14:55:36,255 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/3fd9f8d6ff3643f8909ccfb3aca63f11 2023-02-09 14:55:36,255 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/528921d6137d4fdb9008c9c7fd2bd812 2023-02-09 14:55:36,669 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020911b67faf7fa545c1b26fe0e0b852460c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020911b67faf7fa545c1b26fe0e0b852460c 2023-02-09 14:55:36,672 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209feb1a63defe4480b992796521e3f96c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209feb1a63defe4480b992796521e3f96c0 2023-02-09 14:55:36,677 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209de9df3754b524d859a848070a668493a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209de9df3754b524d859a848070a668493a 2023-02-09 14:55:37,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/af5773a7bcbb460aa26586726044e40f 2023-02-09 14:55:37,078 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/528921d6137d4fdb9008c9c7fd2bd812 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812 2023-02-09 14:55:37,079 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/916b8fd9cc1b48e29ec6e4c381e51ba2 2023-02-09 14:55:37,081 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/b80e5747215a4a88a1a97c0e4173f7ba 2023-02-09 14:55:37,082 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:37,082 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/af5773a7bcbb460aa26586726044e40f as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f 2023-02-09 14:55:37,082 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/3fd9f8d6ff3643f8909ccfb3aca63f11 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11 2023-02-09 14:55:37,086 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/20fdafcb084e4708952478593c937eaa as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa 2023-02-09 14:55:37,087 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:37,087 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:37,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/916b8fd9cc1b48e29ec6e4c381e51ba2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2 2023-02-09 14:55:37,088 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a92002b17d4c5e771b3092e4c4816fd2 in 1652ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:37,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:37,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:37,088 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:37,090 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 14:55:37,091 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/b80e5747215a4a88a1a97c0e4173f7ba as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba 2023-02-09 14:55:37,091 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:37,092 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9a00c85257e134deaed37338dc2b5000 in 1655ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:37,092 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:37,092 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:37,092 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:37,094 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:55:37,095 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 003f0df30fea469fcf63c1030019683b in 1658ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:37,095 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:37,095 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 14:55:37,095 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 14:55:37,097 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,097 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,097 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,097 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 14:55:37,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@601473cc[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:37,099 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 14:55:37,099 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,101 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,102 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 14:55:37,102 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 14:55:37,102 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:37,102 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,102 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:37,106 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 14:55:37,106 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,106 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,106 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 14:55:37,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 14:55:37,108 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 14:55:37,108 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:37,109 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:37,109 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 14:55:37,109 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:37,110 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,110 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,110 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,110 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,111 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:37,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,115 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,115 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 14:55:37,115 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:37,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,115 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:37,216 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:37,216 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,216 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:37,217 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:37,218 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:37,218 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:37,219 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,220 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,220 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:37,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:37,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 14:55:37,222 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 14:55:37,222 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,222 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 14:55:37,222 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 14:55:37,222 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,222 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:37,224 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,224 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,224 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:37,224 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:37,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:37,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:37,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:37,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 14:55:37,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 14:55:37,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 14:55:37,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 14:55:37,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 14:55:37,225 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:37,225 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. started... 2023-02-09 14:55:37,225 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:37,225 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. started... 2023-02-09 14:55:37,225 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92002b17d4c5e771b3092e4c4816fd2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:37,226 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:37,226 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:37,226 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9a00c85257e134deaed37338dc2b5000 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:37,226 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. started... 2023-02-09 14:55:37,226 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:37,226 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 003f0df30fea469fcf63c1030019683b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:55:37,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302099ae8e1a7d15240458a21be5c87871c73 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302099ae8e1a7d15240458a21be5c87871c73 2023-02-09 14:55:37,642 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209b68f610c960e45e3afa21ce5ab5ad0f7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209b68f610c960e45e3afa21ce5ab5ad0f7 2023-02-09 14:55:37,643 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d622ae3f9aaf47afa42c40776de099c7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209d622ae3f9aaf47afa42c40776de099c7 2023-02-09 14:55:38,057 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/5adf18c0bc4942068e252175794dc5e2 2023-02-09 14:55:38,060 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/add4787baafb4a8f9f57eb484df211bd 2023-02-09 14:55:38,072 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/702a14abcd004feca6fc21c609ec4c3a 2023-02-09 14:55:38,478 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209a25bee622e554ac2b0a5a57f13b467cc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209a25bee622e554ac2b0a5a57f13b467cc 2023-02-09 14:55:38,479 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020942d44585198645debb7094f541c1aadc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023020942d44585198645debb7094f541c1aadc 2023-02-09 14:55:38,483 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/906ac0e88d0e4edaae8f62006c95b931 2023-02-09 14:55:38,484 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/a5ee737fd2eb490298b52a4a32a1dc45 2023-02-09 14:55:38,485 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209b8288fe0746a410ea9746edbf7840788 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209b8288fe0746a410ea9746edbf7840788 2023-02-09 14:55:38,487 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/5adf18c0bc4942068e252175794dc5e2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2 2023-02-09 14:55:38,489 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/add4787baafb4a8f9f57eb484df211bd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd 2023-02-09 14:55:38,490 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/e487ffe486dd4045b7ba2810a9126cec 2023-02-09 14:55:38,491 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:38,492 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/906ac0e88d0e4edaae8f62006c95b931 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931 2023-02-09 14:55:38,493 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:38,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/a5ee737fd2eb490298b52a4a32a1dc45 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45 2023-02-09 14:55:38,494 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/702a14abcd004feca6fc21c609ec4c3a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a 2023-02-09 14:55:38,497 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:38,498 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9a00c85257e134deaed37338dc2b5000 in 1272ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:38,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:38,499 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:38,499 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:38,499 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 14:55:38,500 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/e487ffe486dd4045b7ba2810a9126cec as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec 2023-02-09 14:55:38,500 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:38,501 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 003f0df30fea469fcf63c1030019683b in 1275ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:38,501 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:38,501 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:38,501 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:38,504 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:55:38,504 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a92002b17d4c5e771b3092e4c4816fd2 in 1279ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:38,504 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:38,504 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:38,504 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 14:55:38,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 14:55:38,506 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,506 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,506 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,507 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 14:55:38,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15dc31bc[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:38,508 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 14:55:38,508 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,510 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,511 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 14:55:38,511 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 14:55:38,511 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:38,511 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,511 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:38,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 14:55:38,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,512 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,512 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 14:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:38,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 14:55:38,515 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 14:55:38,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:38,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:38,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 14:55:38,515 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:38,516 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,516 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,516 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,517 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,521 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:38,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:38,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 14:55:38,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:38,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,523 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:38,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 14:55:38,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:38,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,624 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:38,624 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,624 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:38,626 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:38,626 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:38,626 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:55:38,628 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,628 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:38,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:38,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 14:55:38,630 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 14:55:38,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 14:55:38,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 14:55:38,631 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,631 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:38,638 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,638 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,638 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:38,638 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:38,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:38,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:38,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:38,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 14:55:38,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 14:55:38,639 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 14:55:38,639 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 14:55:38,639 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,639 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 14:55:38,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:38,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. started... 2023-02-09 14:55:38,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:38,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. started... 2023-02-09 14:55:38,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92002b17d4c5e771b3092e4c4816fd2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:38,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9a00c85257e134deaed37338dc2b5000 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:38,640 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:38,640 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:38,640 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:38,640 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. started... 2023-02-09 14:55:38,640 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 003f0df30fea469fcf63c1030019683b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:55:38,779 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:39,050 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209dc3136ac42084325b52daf16eee25e77 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209dc3136ac42084325b52daf16eee25e77 2023-02-09 14:55:39,051 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302097d0a25350b2c4f15a016712180506ced to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302097d0a25350b2c4f15a016712180506ced 2023-02-09 14:55:39,051 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d1c3af60e0ed43e89593b9aaef555ca6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209d1c3af60e0ed43e89593b9aaef555ca6 2023-02-09 14:55:39,455 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/4d5377fe2b6b49d2a716353f0d9fb081 2023-02-09 14:55:39,455 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/79062239ade3443493acb288f08dcb56 2023-02-09 14:55:39,456 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/0a5161d626df42c48289aca158fd265e 2023-02-09 14:55:39,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209fe57ec85ac09444fb906d0fbff2e9376 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209fe57ec85ac09444fb906d0fbff2e9376 2023-02-09 14:55:39,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020972b0b67b1fd149138037172853296c37 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020972b0b67b1fd149138037172853296c37 2023-02-09 14:55:39,874 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f79a2ae8e7c74ffaafef8c70b05ef7c6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209f79a2ae8e7c74ffaafef8c70b05ef7c6 2023-02-09 14:55:39,882 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/77baa85f4c3e4fdd858dde736e4d60f8 2023-02-09 14:55:39,885 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/79062239ade3443493acb288f08dcb56 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56 2023-02-09 14:55:39,888 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:39,889 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/77baa85f4c3e4fdd858dde736e4d60f8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8 2023-02-09 14:55:39,892 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:39,893 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a92002b17d4c5e771b3092e4c4816fd2 in 1254ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:39,893 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:39,893 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:39,893 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:40,280 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/203911b752aa4d2b9d66d315d8505864 2023-02-09 14:55:40,280 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/8b22f88e690541618f18f85528eb7743 2023-02-09 14:55:40,284 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/4d5377fe2b6b49d2a716353f0d9fb081 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081 2023-02-09 14:55:40,284 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/0a5161d626df42c48289aca158fd265e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e 2023-02-09 14:55:40,287 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:40,287 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 14:55:40,288 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/203911b752aa4d2b9d66d315d8505864 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864 2023-02-09 14:55:40,288 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/8b22f88e690541618f18f85528eb7743 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743 2023-02-09 14:55:40,291 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:40,292 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:55:40,292 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9a00c85257e134deaed37338dc2b5000 in 1653ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:40,292 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:40,292 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:40,292 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 003f0df30fea469fcf63c1030019683b in 1652ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:40,292 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:40,292 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 14:55:40,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 14:55:40,294 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,294 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,294 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,294 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 14:55:40,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fc5efed[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:40,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 14:55:40,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,296 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,296 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 14:55:40,297 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 14:55:40,297 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,297 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:40,297 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:40,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 14:55:40,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,298 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,298 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:40,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 14:55:40,300 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 14:55:40,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:40,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:40,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 14:55:40,300 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:40,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,303 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,304 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,306 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 14:55:40,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:40,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:40,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,306 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,406 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:40,406 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,407 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:40,418 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:40,429 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:40,436 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:40,437 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:40,461 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,462 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,462 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:40,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:40,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 14:55:40,463 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 14:55:40,464 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,464 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 14:55:40,464 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 14:55:40,464 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,464 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:40,465 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,465 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,465 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:40,465 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:40,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:40,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:40,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:40,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 14:55:40,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 14:55:40,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 14:55:40,466 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 14:55:40,466 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,466 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 14:55:40,466 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:40,466 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. started... 2023-02-09 14:55:40,467 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:40,467 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:55:40,467 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:40,467 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92002b17d4c5e771b3092e4c4816fd2 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:40,467 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. started... 2023-02-09 14:55:40,467 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:40,467 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. started... 2023-02-09 14:55:40,467 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 003f0df30fea469fcf63c1030019683b 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:40,467 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9a00c85257e134deaed37338dc2b5000 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:55:40,887 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/bd254de839a54cd8b8a2b77c413a254b 2023-02-09 14:55:40,887 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/ffebdfd165144489ac290069908dafa5 2023-02-09 14:55:40,890 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd254de839a54cd8b8a2b77c413a254b 2023-02-09 14:55:40,891 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffebdfd165144489ac290069908dafa5 2023-02-09 14:55:41,285 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/99313bd0439e4055857c900ec8fd30d6 2023-02-09 14:55:41,289 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99313bd0439e4055857c900ec8fd30d6 2023-02-09 14:55:41,701 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/63565b23f9064e6e8d19f734c3d5dfe9 2023-02-09 14:55:41,702 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/0b8a2f84928b452ba1c9555fbf9dd7c4 2023-02-09 14:55:41,705 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63565b23f9064e6e8d19f734c3d5dfe9 2023-02-09 14:55:41,705 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/bd254de839a54cd8b8a2b77c413a254b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b 2023-02-09 14:55:41,706 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b8a2f84928b452ba1c9555fbf9dd7c4 2023-02-09 14:55:41,706 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/ffebdfd165144489ac290069908dafa5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5 2023-02-09 14:55:41,709 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd254de839a54cd8b8a2b77c413a254b 2023-02-09 14:55:41,709 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:41,710 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/63565b23f9064e6e8d19f734c3d5dfe9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9 2023-02-09 14:55:41,710 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffebdfd165144489ac290069908dafa5 2023-02-09 14:55:41,710 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:41,711 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/0b8a2f84928b452ba1c9555fbf9dd7c4 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4 2023-02-09 14:55:41,715 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b8a2f84928b452ba1c9555fbf9dd7c4 2023-02-09 14:55:41,715 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:41,715 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9a00c85257e134deaed37338dc2b5000 in 1248ms, sequenceid=23, compaction requested=true 2023-02-09 14:55:41,716 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:41,716 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:41,716 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:55:41,716 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63565b23f9064e6e8d19f734c3d5dfe9 2023-02-09 14:55:41,716 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:41,717 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 003f0df30fea469fcf63c1030019683b in 1250ms, sequenceid=23, compaction requested=true 2023-02-09 14:55:41,717 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:41,717 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:41,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:55:42,099 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/8314d45ce0ff46368b7ed791e4949f36 2023-02-09 14:55:42,103 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8314d45ce0ff46368b7ed791e4949f36 2023-02-09 14:55:42,104 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/99313bd0439e4055857c900ec8fd30d6 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6 2023-02-09 14:55:42,108 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99313bd0439e4055857c900ec8fd30d6 2023-02-09 14:55:42,108 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:42,108 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/8314d45ce0ff46368b7ed791e4949f36 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36 2023-02-09 14:55:42,112 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8314d45ce0ff46368b7ed791e4949f36 2023-02-09 14:55:42,112 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:55:42,113 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a92002b17d4c5e771b3092e4c4816fd2 in 1646ms, sequenceid=23, compaction requested=true 2023-02-09 14:55:42,113 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:42,113 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 14:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 14:55:42,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,115 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:42,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,115 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 14:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@701795e8[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:42,116 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 14:55:42,116 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,120 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 14:55:42,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 14:55:42,120 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:42,120 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,120 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:42,122 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 14:55:42,122 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:42,122 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,122 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 14:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:42,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 14:55:42,124 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 14:55:42,124 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:42,124 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:42,124 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 14:55:42,124 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:42,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:42,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:42,128 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:42,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:42,129 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:42,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:42,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:42,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,130 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:42,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 14:55:42,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 14:55:42,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:42,132 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 14:55:42,132 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:42,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:42,132 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:42,132 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:42,133 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:42,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 14:55:42,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:42,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:42,233 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:42,233 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:42,233 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:42,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:42,234 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:42,234 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): a92002b17d4c5e771b3092e4c4816fd2/family1 is initiating major compaction (all files) 2023-02-09 14:55:42,235 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:42,235 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:42,236 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of a92002b17d4c5e771b3092e4c4816fd2/family1 in ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:42,236 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp, totalSize=26.6 K 2023-02-09 14:55:42,236 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting d4da09cafeff4e65961ca123e78f6986, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533642 2023-02-09 14:55:42,237 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 528921d6137d4fdb9008c9c7fd2bd812, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:42,237 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 702a14abcd004feca6fc21c609ec4c3a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537218 2023-02-09 14:55:42,238 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 79062239ade3443493acb288f08dcb56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:42,238 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 99313bd0439e4055857c900ec8fd30d6, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:42,248 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): a92002b17d4c5e771b3092e4c4816fd2#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:42,658 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209da51c3ebbc7641729cbdd98c16593fe9_del 2023-02-09 14:55:42,658 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209da51c3ebbc7641729cbdd98c16593fe9_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209da51c3ebbc7641729cbdd98c16593fe9_del 2023-02-09 14:55:43,474 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb518315c8ff4d22b982ca8c366fb547 2023-02-09 14:55:43,475 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family1/cb518315c8ff4d22b982ca8c366fb547 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/cb518315c8ff4d22b982ca8c366fb547 2023-02-09 14:55:43,479 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb518315c8ff4d22b982ca8c366fb547 2023-02-09 14:55:43,480 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a92002b17d4c5e771b3092e4c4816fd2/family1 of a92002b17d4c5e771b3092e4c4816fd2 into cb518315c8ff4d22b982ca8c366fb547(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:43,480 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:43,480 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:43,480 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): a92002b17d4c5e771b3092e4c4816fd2/family2 is initiating major compaction (all files) 2023-02-09 14:55:43,480 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:43,480 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:43,480 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of a92002b17d4c5e771b3092e4c4816fd2/family2 in ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:55:43,480 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp, totalSize=25.3 K 2023-02-09 14:55:43,481 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3fd2063476584601950204ca7ff39e34, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533642 2023-02-09 14:55:43,481 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting af5773a7bcbb460aa26586726044e40f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:43,481 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting e487ffe486dd4045b7ba2810a9126cec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537218 2023-02-09 14:55:43,482 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 77baa85f4c3e4fdd858dde736e4d60f8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:43,482 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 8314d45ce0ff46368b7ed791e4949f36, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:43,489 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): a92002b17d4c5e771b3092e4c4816fd2#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:43,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6] to archive 2023-02-09 14:55:43,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:43,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/d4da09cafeff4e65961ca123e78f6986 2023-02-09 14:55:43,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/528921d6137d4fdb9008c9c7fd2bd812 2023-02-09 14:55:43,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/702a14abcd004feca6fc21c609ec4c3a 2023-02-09 14:55:43,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/79062239ade3443493acb288f08dcb56 2023-02-09 14:55:43,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/99313bd0439e4055857c900ec8fd30d6 2023-02-09 14:55:43,898 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302091cbc22824e0c4c40b4fbabdc5734d3dd_del 2023-02-09 14:55:43,898 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302091cbc22824e0c4c40b4fbabdc5734d3dd_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302091cbc22824e0c4c40b4fbabdc5734d3dd_del 2023-02-09 14:55:44,711 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51e47f07418d45dfbdd4b5dac6b900c4 2023-02-09 14:55:44,712 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/.tmp/family2/51e47f07418d45dfbdd4b5dac6b900c4 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/51e47f07418d45dfbdd4b5dac6b900c4 2023-02-09 14:55:44,715 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51e47f07418d45dfbdd4b5dac6b900c4 2023-02-09 14:55:44,716 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a92002b17d4c5e771b3092e4c4816fd2/family2 of a92002b17d4c5e771b3092e4c4816fd2 into 51e47f07418d45dfbdd4b5dac6b900c4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:44,716 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:44,716 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:44,716 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 9a00c85257e134deaed37338dc2b5000/family1 is initiating major compaction (all files) 2023-02-09 14:55:44,716 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:44,716 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:44,716 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 9a00c85257e134deaed37338dc2b5000/family1 in ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:44,716 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp, totalSize=26.6 K 2023-02-09 14:55:44,717 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting fe4dadceb79643c8adaf250fdef021fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533642 2023-02-09 14:55:44,717 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3fd9f8d6ff3643f8909ccfb3aca63f11, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:44,717 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 5adf18c0bc4942068e252175794dc5e2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537217 2023-02-09 14:55:44,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36] to archive 2023-02-09 14:55:44,718 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 4d5377fe2b6b49d2a716353f0d9fb081, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:44,718 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting ffebdfd165144489ac290069908dafa5, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:44,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:44,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/3fd2063476584601950204ca7ff39e34 2023-02-09 14:55:44,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/af5773a7bcbb460aa26586726044e40f 2023-02-09 14:55:44,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/e487ffe486dd4045b7ba2810a9126cec 2023-02-09 14:55:44,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/77baa85f4c3e4fdd858dde736e4d60f8 2023-02-09 14:55:44,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family2/8314d45ce0ff46368b7ed791e4949f36 2023-02-09 14:55:44,728 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 9a00c85257e134deaed37338dc2b5000#family1#compaction#151 average throughput is 1.65 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:45,135 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302095a29d2370b7943759e44c278362566e5_del 2023-02-09 14:55:45,135 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302095a29d2370b7943759e44c278362566e5_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302095a29d2370b7943759e44c278362566e5_del 2023-02-09 14:55:45,549 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c7e7e2d3ee484cd4a12f8ae7189fb191 2023-02-09 14:55:45,550 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family1/c7e7e2d3ee484cd4a12f8ae7189fb191 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/c7e7e2d3ee484cd4a12f8ae7189fb191 2023-02-09 14:55:45,553 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c7e7e2d3ee484cd4a12f8ae7189fb191 2023-02-09 14:55:45,554 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9a00c85257e134deaed37338dc2b5000/family1 of 9a00c85257e134deaed37338dc2b5000 into c7e7e2d3ee484cd4a12f8ae7189fb191(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:45,554 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:45,554 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:45,554 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 9a00c85257e134deaed37338dc2b5000/family2 is initiating major compaction (all files) 2023-02-09 14:55:45,554 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:45,554 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:45,554 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 9a00c85257e134deaed37338dc2b5000/family2 in ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:55:45,555 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp, totalSize=25.3 K 2023-02-09 14:55:45,555 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3e0e533f69fb49ccadeab586ba5904db, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533642 2023-02-09 14:55:45,555 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 916b8fd9cc1b48e29ec6e4c381e51ba2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:45,556 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 906ac0e88d0e4edaae8f62006c95b931, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537217 2023-02-09 14:55:45,556 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 203911b752aa4d2b9d66d315d8505864, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:45,556 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 0b8a2f84928b452ba1c9555fbf9dd7c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:45,564 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 9a00c85257e134deaed37338dc2b5000#family2#compaction#152 average throughput is 1.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:45,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5] to archive 2023-02-09 14:55:45,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:45,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/fe4dadceb79643c8adaf250fdef021fd 2023-02-09 14:55:45,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/3fd9f8d6ff3643f8909ccfb3aca63f11 2023-02-09 14:55:45,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/5adf18c0bc4942068e252175794dc5e2 2023-02-09 14:55:45,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/4d5377fe2b6b49d2a716353f0d9fb081 2023-02-09 14:55:45,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/ffebdfd165144489ac290069908dafa5 2023-02-09 14:55:45,970 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209c0fbc024ec6b431d913661d8f7651235_del 2023-02-09 14:55:45,971 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209c0fbc024ec6b431d913661d8f7651235_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209c0fbc024ec6b431d913661d8f7651235_del 2023-02-09 14:55:46,785 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76082650108d47b69f953a5ad8928112 2023-02-09 14:55:46,786 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/.tmp/family2/76082650108d47b69f953a5ad8928112 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/76082650108d47b69f953a5ad8928112 2023-02-09 14:55:46,789 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76082650108d47b69f953a5ad8928112 2023-02-09 14:55:46,790 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9a00c85257e134deaed37338dc2b5000/family2 of 9a00c85257e134deaed37338dc2b5000 into 76082650108d47b69f953a5ad8928112(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:46,790 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:46,790 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:46,790 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 003f0df30fea469fcf63c1030019683b/family1 is initiating major compaction (all files) 2023-02-09 14:55:46,790 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:46,790 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:46,790 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 003f0df30fea469fcf63c1030019683b/family1 in ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:46,790 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp, totalSize=26.6 K 2023-02-09 14:55:46,791 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting b6a28187a8e4488ba5901a0b16a60afa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533643 2023-02-09 14:55:46,791 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 20fdafcb084e4708952478593c937eaa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:46,791 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting add4787baafb4a8f9f57eb484df211bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537218 2023-02-09 14:55:46,792 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 0a5161d626df42c48289aca158fd265e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:46,792 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting bd254de839a54cd8b8a2b77c413a254b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:46,799 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 003f0df30fea469fcf63c1030019683b#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:46,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4] to archive 2023-02-09 14:55:46,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:46,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/3e0e533f69fb49ccadeab586ba5904db 2023-02-09 14:55:46,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/916b8fd9cc1b48e29ec6e4c381e51ba2 2023-02-09 14:55:46,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/906ac0e88d0e4edaae8f62006c95b931 2023-02-09 14:55:46,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/203911b752aa4d2b9d66d315d8505864 2023-02-09 14:55:46,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family2/0b8a2f84928b452ba1c9555fbf9dd7c4 2023-02-09 14:55:47,206 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020907edb573dcf646aca975e41e5d024888_del 2023-02-09 14:55:47,207 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020907edb573dcf646aca975e41e5d024888_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023020907edb573dcf646aca975e41e5d024888_del 2023-02-09 14:55:48,020 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37afe8061b764868b4c49a5a0e5c5019 2023-02-09 14:55:48,021 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family1/37afe8061b764868b4c49a5a0e5c5019 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/37afe8061b764868b4c49a5a0e5c5019 2023-02-09 14:55:48,025 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37afe8061b764868b4c49a5a0e5c5019 2023-02-09 14:55:48,026 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 003f0df30fea469fcf63c1030019683b/family1 of 003f0df30fea469fcf63c1030019683b into 37afe8061b764868b4c49a5a0e5c5019(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:48,026 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:48,026 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:55:48,026 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 003f0df30fea469fcf63c1030019683b/family2 is initiating major compaction (all files) 2023-02-09 14:55:48,026 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:55:48,026 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:55:48,026 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 003f0df30fea469fcf63c1030019683b/family2 in ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:55:48,026 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp, totalSize=25.3 K 2023-02-09 14:55:48,027 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 56b77831d2c24f1b8f6588436048263b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954533643 2023-02-09 14:55:48,027 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting b80e5747215a4a88a1a97c0e4173f7ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954535429 2023-02-09 14:55:48,027 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting a5ee737fd2eb490298b52a4a32a1dc45, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954537218 2023-02-09 14:55:48,028 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 8b22f88e690541618f18f85528eb7743, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954538626 2023-02-09 14:55:48,028 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 63565b23f9064e6e8d19f734c3d5dfe9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:55:48,038 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 003f0df30fea469fcf63c1030019683b#family2#compaction#154 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:48,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b] to archive 2023-02-09 14:55:48,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:48,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/b6a28187a8e4488ba5901a0b16a60afa 2023-02-09 14:55:48,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/20fdafcb084e4708952478593c937eaa 2023-02-09 14:55:48,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/add4787baafb4a8f9f57eb484df211bd 2023-02-09 14:55:48,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/0a5161d626df42c48289aca158fd265e 2023-02-09 14:55:48,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/bd254de839a54cd8b8a2b77c413a254b 2023-02-09 14:55:48,446 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020967b946f5bf6543ea815d5e528594b4c5_del 2023-02-09 14:55:48,446 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020967b946f5bf6543ea815d5e528594b4c5_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023020967b946f5bf6543ea815d5e528594b4c5_del 2023-02-09 14:55:49,259 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da73477b2b5e4fc98505cd786ff57aaf 2023-02-09 14:55:49,260 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/.tmp/family2/da73477b2b5e4fc98505cd786ff57aaf as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/da73477b2b5e4fc98505cd786ff57aaf 2023-02-09 14:55:49,263 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da73477b2b5e4fc98505cd786ff57aaf 2023-02-09 14:55:49,264 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 003f0df30fea469fcf63c1030019683b/family2 of 003f0df30fea469fcf63c1030019683b into da73477b2b5e4fc98505cd786ff57aaf(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:55:49,264 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:49,267 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:49,270 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:49,273 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:49,274 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:49,278 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:55:49,290 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-09 14:55:49,292 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302095a29d2370b7943759e44c278362566e5_del 2023-02-09 14:55:49,295 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209da51c3ebbc7641729cbdd98c16593fe9_del 2023-02-09 14:55:49,297 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020907edb573dcf646aca975e41e5d024888_del 2023-02-09 14:55:49,297 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:49,297 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(139): Connect 0x69071657 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:49,302 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3995256a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:49,303 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:49,303 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:49,303 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:49,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9] to archive 2023-02-09 14:55:49,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:49,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/56b77831d2c24f1b8f6588436048263b 2023-02-09 14:55:49,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/b80e5747215a4a88a1a97c0e4173f7ba 2023-02-09 14:55:49,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/a5ee737fd2eb490298b52a4a32a1dc45 2023-02-09 14:55:49,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/8b22f88e690541618f18f85528eb7743 2023-02-09 14:55:49,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family2/63565b23f9064e6e8d19f734c3d5dfe9 2023-02-09 14:55:50,125 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d511d6f5b9854fcbad2c46670f6d0133 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209d511d6f5b9854fcbad2c46670f6d0133 2023-02-09 14:55:50,126 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,133 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209cff9b9aaf47f4406b76f2a473fd7e906 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209cff9b9aaf47f4406b76f2a473fd7e906 2023-02-09 14:55:50,134 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209cbf1f6daad664526b7bead992670f7ea to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230209cbf1f6daad664526b7bead992670f7ea 2023-02-09 14:55:50,134 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,135 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,142 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:50,145 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42180, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:50,153 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,153 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,154 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:55:50,157 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,157 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,157 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40334, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:55:50,158 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,158 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,158 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,158 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,180 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/23a84b5001b344559ebf28de5ecd8433 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-09 14:55:50,180 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-09 14:55:50,181 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/8af6c5c3127d4c9f9a0b3146dff46b3e first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-09 14:55:50,181 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/23a84b5001b344559ebf28de5ecd8433}] 2023-02-09 14:55:50,181 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/3f81b4a8631f4071a1d19ad7e3c2ca88}] 2023-02-09 14:55:50,181 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/8af6c5c3127d4c9f9a0b3146dff46b3e}] 2023-02-09 14:55:50,190 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 for inclusion in 9a00c85257e134deaed37338dc2b5000/family1 2023-02-09 14:55:50,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/8af6c5c3127d4c9f9a0b3146dff46b3e for inclusion in 003f0df30fea469fcf63c1030019683b/family1 2023-02-09 14:55:50,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/23a84b5001b344559ebf28de5ecd8433 for inclusion in a92002b17d4c5e771b3092e4c4816fd2/family1 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 003f0df30fea469fcf63c1030019683b: 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 9a00c85257e134deaed37338dc2b5000: 2023-02-09 14:55:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for a92002b17d4c5e771b3092e4c4816fd2: 2023-02-09 14:55:50,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/8af6c5c3127d4c9f9a0b3146dff46b3e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__mp9nc3nj8o7o89psq1353ner0hcc7tpsdi4uc0462roik3e69219a7ejopj1k1q/family1/8af6c5c3127d4c9f9a0b3146dff46b3e 2023-02-09 14:55:50,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/23a84b5001b344559ebf28de5ecd8433 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__2v3m67u4uuirgd4hh698gjrjej6sr5q3mt42m5catp9j2bjsftde8v5k5hcgghq6/family1/23a84b5001b344559ebf28de5ecd8433 2023-02-09 14:55:50,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__ogvt8c4kredc0j6q5t3eg7bk6s57pab7dnh9nqfdhqq6vklhhun9qvladcvu6op/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 2023-02-09 14:55:50,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__mp9nc3nj8o7o89psq1353ner0hcc7tpsdi4uc0462roik3e69219a7ejopj1k1q/family1/8af6c5c3127d4c9f9a0b3146dff46b3e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/e76d149740c440e7b2cb60bd2437c141_SeqId_28_ 2023-02-09 14:55:50,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__2v3m67u4uuirgd4hh698gjrjej6sr5q3mt42m5catp9j2bjsftde8v5k5hcgghq6/family1/23a84b5001b344559ebf28de5ecd8433 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/2d2c6836d73648e08d779a2f0feb990c_SeqId_28_ 2023-02-09 14:55:50,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__ogvt8c4kredc0j6q5t3eg7bk6s57pab7dnh9nqfdhqq6vklhhun9qvladcvu6op/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/0eb014a8643c4819aa1d52ee0139efab_SeqId_28_ 2023-02-09 14:55:50,198 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__mp9nc3nj8o7o89psq1353ner0hcc7tpsdi4uc0462roik3e69219a7ejopj1k1q/family1/8af6c5c3127d4c9f9a0b3146dff46b3e into 003f0df30fea469fcf63c1030019683b/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/e76d149740c440e7b2cb60bd2437c141_SeqId_28_ - updating store file list. 2023-02-09 14:55:50,199 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__ogvt8c4kredc0j6q5t3eg7bk6s57pab7dnh9nqfdhqq6vklhhun9qvladcvu6op/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 into 9a00c85257e134deaed37338dc2b5000/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/0eb014a8643c4819aa1d52ee0139efab_SeqId_28_ - updating store file list. 2023-02-09 14:55:50,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__2v3m67u4uuirgd4hh698gjrjej6sr5q3mt42m5catp9j2bjsftde8v5k5hcgghq6/family1/23a84b5001b344559ebf28de5ecd8433 into a92002b17d4c5e771b3092e4c4816fd2/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/2d2c6836d73648e08d779a2f0feb990c_SeqId_28_ - updating store file list. 2023-02-09 14:55:50,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/e76d149740c440e7b2cb60bd2437c141_SeqId_28_ into 003f0df30fea469fcf63c1030019683b/family1 2023-02-09 14:55:50,202 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__mp9nc3nj8o7o89psq1353ner0hcc7tpsdi4uc0462roik3e69219a7ejopj1k1q/family1/8af6c5c3127d4c9f9a0b3146dff46b3e into 003f0df30fea469fcf63c1030019683b/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/family1/e76d149740c440e7b2cb60bd2437c141_SeqId_28_) 2023-02-09 14:55:50,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__mp9nc3nj8o7o89psq1353ner0hcc7tpsdi4uc0462roik3e69219a7ejopj1k1q/family1/8af6c5c3127d4c9f9a0b3146dff46b3e 2023-02-09 14:55:50,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/2d2c6836d73648e08d779a2f0feb990c_SeqId_28_ into a92002b17d4c5e771b3092e4c4816fd2/family1 2023-02-09 14:55:50,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__2v3m67u4uuirgd4hh698gjrjej6sr5q3mt42m5catp9j2bjsftde8v5k5hcgghq6/family1/23a84b5001b344559ebf28de5ecd8433 into a92002b17d4c5e771b3092e4c4816fd2/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/family1/2d2c6836d73648e08d779a2f0feb990c_SeqId_28_) 2023-02-09 14:55:50,203 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/0eb014a8643c4819aa1d52ee0139efab_SeqId_28_ into 9a00c85257e134deaed37338dc2b5000/family1 2023-02-09 14:55:50,204 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__ogvt8c4kredc0j6q5t3eg7bk6s57pab7dnh9nqfdhqq6vklhhun9qvladcvu6op/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 into 9a00c85257e134deaed37338dc2b5000/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/family1/0eb014a8643c4819aa1d52ee0139efab_SeqId_28_) 2023-02-09 14:55:50,205 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__2v3m67u4uuirgd4hh698gjrjej6sr5q3mt42m5catp9j2bjsftde8v5k5hcgghq6/family1/23a84b5001b344559ebf28de5ecd8433 2023-02-09 14:55:50,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__ns_testMinorCompaction__ogvt8c4kredc0j6q5t3eg7bk6s57pab7dnh9nqfdhqq6vklhhun9qvladcvu6op/family1/3f81b4a8631f4071a1d19ad7e3c2ca88 2023-02-09 14:55:50,205 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:50,207 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,207 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,207 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:50,208 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302090aa1cc13521649eeb9026973d26c9a87 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302090aa1cc13521649eeb9026973d26c9a87 2023-02-09 14:55:50,208 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 14:55:50,208 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 14:55:50,208 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:50,208 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:50,209 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982ffaf72f2cd482d89bc9edf1c203584 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020982ffaf72f2cd482d89bc9edf1c203584 2023-02-09 14:55:50,209 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 14:55:50,209 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 14:55:50,209 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:50,209 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302095ca4ef6e0c54476bb0961cced99d3e87 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302095ca4ef6e0c54476bb0961cced99d3e87 2023-02-09 14:55:50,210 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 14:55:50,210 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 14:55:50,210 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 14:55:50,210 INFO [Listener at localhost/33189] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-09 14:55:50,212 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:50,214 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:50,216 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:55:50,217 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:55:50,235 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=255 (was 285), OpenFileDescriptor=649 (was 764), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 99), ProcessCount=168 (was 168), AvailableMemoryMB=2941 (was 2983) 2023-02-09 14:55:50,247 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255, OpenFileDescriptor=649, MaxFileDescriptor=60000, SystemLoadAverage=86, ProcessCount=168, AvailableMemoryMB=2941 2023-02-09 14:55:50,247 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,247 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,249 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 14:55:50,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,251 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:50,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-09 14:55:50,251 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,251 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,251 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:50,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 14:55:50,253 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,253 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4 empty. 2023-02-09 14:55:50,253 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,253 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-09 14:55:50,260 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:50,261 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4a620fe72a6269f78f560319e3947df4, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:55:50,265 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:50,265 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 4a620fe72a6269f78f560319e3947df4, disabling compactions & flushes 2023-02-09 14:55:50,265 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,265 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,266 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. after waiting 0 ms 2023-02-09 14:55:50,266 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,266 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,266 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 4a620fe72a6269f78f560319e3947df4: Waiting for close lock at 1675954550265Disabling compacts and flushes for region at 1675954550265Disabling writes for close at 1675954550266 (+1 ms)Writing region close event to WAL at 1675954550266Closed at 1675954550266 2023-02-09 14:55:50,268 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:50,268 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954550268"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954550268"}]},"ts":"1675954550268"} 2023-02-09 14:55:50,270 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:50,270 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:50,270 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954550270"}]},"ts":"1675954550270"} 2023-02-09 14:55:50,271 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 14:55:50,274 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=4a620fe72a6269f78f560319e3947df4, ASSIGN}] 2023-02-09 14:55:50,275 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=4a620fe72a6269f78f560319e3947df4, ASSIGN 2023-02-09 14:55:50,276 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=4a620fe72a6269f78f560319e3947df4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:55:50,352 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,352 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 14:55:50,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=4a620fe72a6269f78f560319e3947df4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:50,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954550427"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954550427"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954550427"}]},"ts":"1675954550427"} 2023-02-09 14:55:50,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 4a620fe72a6269f78f560319e3947df4, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:55:50,553 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,554 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 14:55:50,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4a620fe72a6269f78f560319e3947df4, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:50,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:50,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,584 INFO [StoreOpener-4a620fe72a6269f78f560319e3947df4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,585 DEBUG [StoreOpener-4a620fe72a6269f78f560319e3947df4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:50,585 DEBUG [StoreOpener-4a620fe72a6269f78f560319e3947df4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:50,586 INFO [StoreOpener-4a620fe72a6269f78f560319e3947df4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4a620fe72a6269f78f560319e3947df4 columnFamilyName family1 2023-02-09 14:55:50,587 INFO [StoreOpener-4a620fe72a6269f78f560319e3947df4-1] regionserver.HStore(310): Store=4a620fe72a6269f78f560319e3947df4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:50,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:55:50,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:50,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4a620fe72a6269f78f560319e3947df4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59986745, jitterRate=-0.10612784326076508}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:55:50,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4a620fe72a6269f78f560319e3947df4: Running coprocessor pre-open hook at 1675954550583Writing region info on filesystem at 1675954550583Initializing all the Stores at 1675954550584 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675954550584Cleaning up temporary data from old regions at 1675954550588 (+4 ms)Cleaning up detritus from prior splits at 1675954550588Running coprocessor post-open hooks at 1675954550591 (+3 ms)Region opened successfully at 1675954550591 2023-02-09 14:55:50,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., pid=114, masterSystemTime=1675954550580 2023-02-09 14:55:50,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,593 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=4a620fe72a6269f78f560319e3947df4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:50,593 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954550593"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954550593"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954550593"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954550593"}]},"ts":"1675954550593"} 2023-02-09 14:55:50,595 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 14:55:50,596 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 4a620fe72a6269f78f560319e3947df4, server=jenkins-hbase4.apache.org,43455,1675954416872 in 165 msec 2023-02-09 14:55:50,597 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 14:55:50,597 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=4a620fe72a6269f78f560319e3947df4, ASSIGN in 322 msec 2023-02-09 14:55:50,597 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:50,597 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954550597"}]},"ts":"1675954550597"} 2023-02-09 14:55:50,598 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 14:55:50,600 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:50,601 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 351 msec 2023-02-09 14:55:50,817 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:50,854 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,855 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 14:55:50,856 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-09 14:55:50,876 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,876 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,876 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:50,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,878 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:50,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:50,878 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:50,879 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,879 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,880 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:50,880 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,880 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:50,880 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:50,896 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:50,896 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:50,896 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:50,896 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:50,897 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 14:55:51,325 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130618dc89e35e94875b98b83533b70ea0b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130618dc89e35e94875b98b83533b70ea0b 2023-02-09 14:55:51,729 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/e33605916e8249d4989182d3a9a4f8b0 2023-02-09 14:55:51,733 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/e33605916e8249d4989182d3a9a4f8b0 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0 2023-02-09 14:55:51,737 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 14:55:51,738 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 840ms, sequenceid=5, compaction requested=false 2023-02-09 14:55:51,738 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:51,738 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,740 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:51,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@335e1235[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:51,741 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:51,741 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,745 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:51,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:51,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,746 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:51,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:51,747 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:51,747 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,747 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,747 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:51,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:51,749 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:51,749 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:51,749 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:51,749 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,749 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:51,751 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,751 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,751 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,752 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,753 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,755 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,755 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,755 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,755 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:51,756 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:51,755 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,755 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:51,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,756 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,856 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:51,856 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,856 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:51,860 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,860 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:51,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:51,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,862 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:51,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:51,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:51,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,862 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,864 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,864 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,865 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:51,865 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,865 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,866 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:51,866 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:51,866 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,866 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:51,866 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:51,866 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:51,866 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:51,866 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:51,866 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 14:55:52,277 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287b2b1a10098f472db5b90d2ef7e40415 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511287b2b1a10098f472db5b90d2ef7e40415 2023-02-09 14:55:52,681 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/e1aa30069b694a7484edba5e3c9e2807 2023-02-09 14:55:52,685 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/e1aa30069b694a7484edba5e3c9e2807 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807 2023-02-09 14:55:52,688 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 14:55:52,689 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 823ms, sequenceid=9, compaction requested=false 2023-02-09 14:55:52,689 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:52,689 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:52,689 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:52,689 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:52,689 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,689 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:52,689 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:52,691 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,691 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:52,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:52,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@477b9433[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:52,692 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:52,692 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,693 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,693 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:52,693 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:52,693 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:52,694 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,694 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:52,694 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:52,695 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:52,694 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,695 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:52,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:52,697 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:52,697 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:52,697 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:52,697 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,697 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:52,699 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:52,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:52,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:52,699 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,700 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:52,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,702 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:52,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:52,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:52,702 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,703 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,703 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:52,703 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:52,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:52,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:52,803 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:52,803 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,803 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:52,806 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,807 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,807 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:52,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:52,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,809 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:52,809 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:52,809 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:52,809 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:52,809 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,809 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,814 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:52,814 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:52,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:52,815 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:52,815 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:52,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:52,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:52,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:52,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:52,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:52,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:52,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:52,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:52,816 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:52,816 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:52,816 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:53,226 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201feb58f9214204ea7808253b6430861d8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201feb58f9214204ea7808253b6430861d8 2023-02-09 14:55:53,631 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/be64605936f64a71aaac547f2b270eb8 2023-02-09 14:55:53,636 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/be64605936f64a71aaac547f2b270eb8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8 2023-02-09 14:55:53,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 14:55:53,640 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 824ms, sequenceid=13, compaction requested=true 2023-02-09 14:55:53,640 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:53,640 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:53,640 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:53,640 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:53,640 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,641 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:53,641 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:53,642 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,642 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:53,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:53,642 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,642 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:53,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:53,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:53,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:53,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:53,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7771a1b6[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:53,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:53,644 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,645 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,645 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:53,645 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:53,645 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:53,645 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,646 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:53,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:53,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:53,647 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,647 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:53,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:53,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:53,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:53,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:53,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:53,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:53,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:53,650 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:53,650 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:53,650 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:53,650 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,650 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:53,652 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,652 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:53,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,652 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:53,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:53,652 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:53,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:53,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:53,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,656 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:53,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:53,659 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:53,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:53,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:53,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:53,659 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:53,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:53,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:53,760 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:53,760 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,760 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:53,764 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,764 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:53,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:53,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,766 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:53,766 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:53,766 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:53,766 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:53,766 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,766 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,767 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:53,767 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:53,768 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:53,768 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:53,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:53,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:53,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:53,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:53,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:53,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:53,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:53,769 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:53,769 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:53,769 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:53,769 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:54,179 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052ce79179a35840b7b100f8b998dc2b3d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512052ce79179a35840b7b100f8b998dc2b3d 2023-02-09 14:55:54,584 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/7dd0879e0dff4bd99b7c577741343c29 2023-02-09 14:55:54,588 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/7dd0879e0dff4bd99b7c577741343c29 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29 2023-02-09 14:55:54,592 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 14:55:54,593 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 823ms, sequenceid=17, compaction requested=true 2023-02-09 14:55:54,593 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:54,593 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:54,593 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:54,593 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:54,593 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,593 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:54,593 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:54,595 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,595 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:54,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:54,595 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,595 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:54,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:54,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:54,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:54,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:54,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b85d265[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:54,597 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:54,597 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,598 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,598 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:54,598 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:54,598 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:54,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,598 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:54,599 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:54,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:54,601 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:54,601 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:54,601 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:54,601 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,601 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:54,603 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,603 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:54,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:54,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,603 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:54,604 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,605 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:54,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,607 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:54,607 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:54,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:54,607 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:54,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:54,608 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:54,607 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:54,607 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:54,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,608 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:54,708 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:54,708 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:54,708 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:54,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:54,712 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:54,712 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:54,712 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:54,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:54,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,714 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:54,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:54,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:54,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:54,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,714 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,716 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:54,716 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:54,716 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:54,716 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:54,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,717 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,717 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:54,717 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:54,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:54,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:54,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:54,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:54,717 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:54,717 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:54,717 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:54,717 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:55,127 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122826e231f99d45459a9bef90026c4ceae6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122826e231f99d45459a9bef90026c4ceae6 2023-02-09 14:55:55,532 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/d7e39efe925943d5862df93a58c8cb06 2023-02-09 14:55:55,536 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/d7e39efe925943d5862df93a58c8cb06 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06 2023-02-09 14:55:55,539 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 14:55:55,540 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 823ms, sequenceid=21, compaction requested=true 2023-02-09 14:55:55,540 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:55,540 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:55,542 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,542 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:55,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:55,543 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:55,543 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:55,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:55,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7889b09a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:55,544 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:55,544 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,545 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,545 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:55,545 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:55,545 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,545 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:55,545 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:55,548 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:55,548 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:55,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:55,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:55,550 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:55,550 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:55,550 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:55,550 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,550 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:55,551 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,551 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:55,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:55,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,551 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:55,552 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,553 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:55,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,558 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,559 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:55,559 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:55,559 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:55,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:55,559 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,559 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:55,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:55,659 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:55,660 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:55,660 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:55,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:55,663 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:55,663 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:55,664 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:55,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:55,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,665 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:55,665 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:55,666 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:55,666 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:55,666 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,666 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,667 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:55,667 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:55,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:55,667 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:55,667 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:55,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:55,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:55,668 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:55,668 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:55,668 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:55,668 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:55,668 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:55,669 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:55,669 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:55,669 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:56,080 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316232e6ec0ade40238dc01e4059796386 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316232e6ec0ade40238dc01e4059796386 2023-02-09 14:55:56,118 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:56,486 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/dd5a46f430604f06bda85e45c316099e 2023-02-09 14:55:56,498 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/dd5a46f430604f06bda85e45c316099e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e 2023-02-09 14:55:56,502 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 14:55:56,502 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 833ms, sequenceid=25, compaction requested=true 2023-02-09 14:55:56,503 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:56,503 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:56,503 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:56,503 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:56,503 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,503 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:56,503 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:56,504 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,504 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:56,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,505 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:56,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:56,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:56,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:56,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:56,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63c869b4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:56,506 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:56,506 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,507 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,507 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:56,507 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:56,508 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:56,508 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,508 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:56,510 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,510 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:56,510 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:56,510 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:56,512 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:56,512 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:56,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:56,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:56,513 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,513 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:56,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:56,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,514 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:56,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:56,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:56,515 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:56,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,516 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:56,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:56,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:56,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:56,518 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:56,518 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:56,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:56,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,519 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,519 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,519 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:56,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:56,520 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:56,520 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:56,620 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:56,620 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:56,621 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:56,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:56,625 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:56,625 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:56,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:56,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:56,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,628 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:56,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:56,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:56,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:56,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,628 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,630 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:56,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:56,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:56,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:56,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:56,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:56,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:56,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:56,631 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:56,631 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:56,631 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:56,632 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:56,632 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:56,632 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:56,632 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:56,632 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:57,046 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601011b1628b4dfa549c3b5c060478bdb1824 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b1628b4dfa549c3b5c060478bdb1824 2023-02-09 14:55:57,057 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/616ff826959a4262a04183bb8bc2e108 2023-02-09 14:55:57,062 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/616ff826959a4262a04183bb8bc2e108 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108 2023-02-09 14:55:57,066 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 14:55:57,067 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 435ms, sequenceid=29, compaction requested=true 2023-02-09 14:55:57,067 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:57,067 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:57,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:57,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:57,067 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:57,070 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19ce186a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:57,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:57,072 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,073 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,073 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:57,073 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:57,073 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:57,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,073 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:57,074 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:57,075 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:57,077 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:57,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:57,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:57,077 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,078 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:57,079 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,079 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,079 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,079 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,080 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,084 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,084 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:57,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:57,085 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:57,085 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:57,185 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:57,185 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,185 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:57,189 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,190 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:57,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:57,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,192 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,192 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,192 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:57,192 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:57,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,193 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,194 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,194 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,194 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,194 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:57,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:57,196 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:57,196 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,196 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:57,196 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:57,209 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103137576d12cef4d5fa56d4d239c8882da to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137576d12cef4d5fa56d4d239c8882da 2023-02-09 14:55:57,214 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/7519da7a8cd64c4a98a151f45df7e1b1 2023-02-09 14:55:57,218 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/7519da7a8cd64c4a98a151f45df7e1b1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1 2023-02-09 14:55:57,222 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 14:55:57,223 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 27ms, sequenceid=33, compaction requested=true 2023-02-09 14:55:57,223 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:57,223 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:57,223 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,225 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:57,225 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:57,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c4de7cc[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:57,227 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:57,227 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,228 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:57,229 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,229 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:57,231 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,231 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:57,234 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:57,234 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:57,234 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:57,234 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,234 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:57,236 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,236 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,236 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,244 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,246 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,255 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:57,255 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,255 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:57,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,256 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:57,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,356 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:57,356 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,356 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:57,361 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,361 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,362 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:57,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:57,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,363 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,363 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,364 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:57,364 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:57,364 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,364 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,367 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,367 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,367 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,367 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:57,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:57,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:57,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,368 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:57,368 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,369 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:57,369 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:57,369 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:57,369 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,369 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:57,369 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:57,386 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907003c6ad07b274fd9823bb5579e83142d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907003c6ad07b274fd9823bb5579e83142d 2023-02-09 14:55:57,799 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/4106f99dee834f849c25857fca0387d5 2023-02-09 14:55:57,804 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/4106f99dee834f849c25857fca0387d5 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5 2023-02-09 14:55:57,808 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 14:55:57,809 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 440ms, sequenceid=37, compaction requested=true 2023-02-09 14:55:57,809 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:57,809 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,809 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:57,810 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:57,810 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,810 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:57,810 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:57,811 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,811 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,812 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,812 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:57,814 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39758bbf[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:57,814 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:57,814 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,815 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,815 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:57,815 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:57,815 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,815 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:57,815 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:57,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:57,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,816 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,816 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:57,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:57,819 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:57,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:57,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:57,819 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,819 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:57,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,822 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,823 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,824 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:57,826 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,827 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,826 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,827 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:57,827 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:57,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,927 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:57,927 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,928 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:57,932 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:57,932 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:57,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:57,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:57,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,934 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,934 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,935 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 14:55:57,935 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 14:55:57,935 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,935 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,936 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,936 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,937 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,937 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:57,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:57,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 14:55:57,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 14:55:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 14:55:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:57,938 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:57,938 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,938 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. started... 2023-02-09 14:55:57,938 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4a620fe72a6269f78f560319e3947df4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 14:55:57,952 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112041a96f5d52c34578ae823dfb246a99a0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112041a96f5d52c34578ae823dfb246a99a0 2023-02-09 14:55:57,957 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/f42b1fed00ae46309847737cdcc6b7e8 2023-02-09 14:55:57,961 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/f42b1fed00ae46309847737cdcc6b7e8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8 2023-02-09 14:55:57,964 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 14:55:57,965 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4a620fe72a6269f78f560319e3947df4 in 27ms, sequenceid=41, compaction requested=true 2023-02-09 14:55:57,965 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:57,965 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 14:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 14:55:57,967 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,967 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 14:55:57,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1950c0ec[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:57,969 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 14:55:57,969 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,971 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,972 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 14:55:57,972 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 14:55:57,972 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:55:57,972 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,972 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:57,973 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,973 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 14:55:57,973 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,973 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 14:55:57,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 14:55:57,976 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 14:55:57,976 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:57,976 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:57,976 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,976 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:57,977 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,977 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,977 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:57,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:57,978 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,979 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,982 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:57,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 14:55:57,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 14:55:57,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:55:57,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:57,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:57,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:57,982 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:57,983 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:58,083 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:58,083 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:58,083 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:58,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:58,085 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-09 14:55:58,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:55:58,087 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:55:58,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:55:58,088 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955158088 2023-02-09 14:55:58,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 14:55:58,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 14:55:58,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:55:58,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x5e663a06 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:55:58,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bfc9f5b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:55:58,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:55:58,548 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e6594cbd6c3849809fbcf2172cd22f76 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103e6594cbd6c3849809fbcf2172cd22f76 2023-02-09 14:55:58,549 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:55:58,569 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:58,572 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45220, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:58,582 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:58,582 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:58,584 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:55:58,585 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59716, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:55:58,596 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/62d838a63bf346e39e1a1c8a89dad6dc first=Optional[r5] last=Optional[r6] 2023-02-09 14:55:58,597 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/62d838a63bf346e39e1a1c8a89dad6dc}] 2023-02-09 14:55:58,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/62d838a63bf346e39e1a1c8a89dad6dc for inclusion in 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:58,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 14:55:58,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:55:58,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:58,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/62d838a63bf346e39e1a1c8a89dad6dc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__a6offt29flh3q15glkfipvqtj818kft77ocmkh4k5qelledqo0b3l6pgrk76mpua/family1/62d838a63bf346e39e1a1c8a89dad6dc 2023-02-09 14:55:58,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__a6offt29flh3q15glkfipvqtj818kft77ocmkh4k5qelledqo0b3l6pgrk76mpua/family1/62d838a63bf346e39e1a1c8a89dad6dc as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_ 2023-02-09 14:55:58,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__a6offt29flh3q15glkfipvqtj818kft77ocmkh4k5qelledqo0b3l6pgrk76mpua/family1/62d838a63bf346e39e1a1c8a89dad6dc into 4a620fe72a6269f78f560319e3947df4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_ - updating store file list. 2023-02-09 14:55:58,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_ into 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:58,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__a6offt29flh3q15glkfipvqtj818kft77ocmkh4k5qelledqo0b3l6pgrk76mpua/family1/62d838a63bf346e39e1a1c8a89dad6dc into 4a620fe72a6269f78f560319e3947df4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_) 2023-02-09 14:55:58,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__a6offt29flh3q15glkfipvqtj818kft77ocmkh4k5qelledqo0b3l6pgrk76mpua/family1/62d838a63bf346e39e1a1c8a89dad6dc 2023-02-09 14:55:58,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 14:55:58,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 14:55:58,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.HStore(1912): 4a620fe72a6269f78f560319e3947df4/family1 is initiating minor compaction 2023-02-09 14:55:58,625 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4a620fe72a6269f78f560319e3947df4/family1 in testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:58,625 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp, totalSize=49.7 K 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting e1aa30069b694a7484edba5e3c9e2807, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting be64605936f64a71aaac547f2b270eb8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 7dd0879e0dff4bd99b7c577741343c29, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting d7e39efe925943d5862df93a58c8cb06, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting dd5a46f430604f06bda85e45c316099e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 616ff826959a4262a04183bb8bc2e108, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 7519da7a8cd64c4a98a151f45df7e1b1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 4106f99dee834f849c25857fca0387d5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting f42b1fed00ae46309847737cdcc6b7e8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 14:55:58,626 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 63226b643b404ded95ab5bf4ff4cd786_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 14:55:58,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4d6d5a5b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:58,630 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:55:58,631 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:58,633 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b1628b4dfa549c3b5c060478bdb1824 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601011b1628b4dfa549c3b5c060478bdb1824 2023-02-09 14:55:58,635 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137576d12cef4d5fa56d4d239c8882da to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137576d12cef4d5fa56d4d239c8882da 2023-02-09 14:55:58,635 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 14:55:58,635 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 14:55:58,636 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:55:58,636 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:55:58,703 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4a620fe72a6269f78f560319e3947df4#family1#compaction#165 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:58,795 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:58,795 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0960de82 to 127.0.0.1:56347 2023-02-09 14:55:58,795 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:58,804 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:58,804 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69071657 to 127.0.0.1:56347 2023-02-09 14:55:58,804 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:59,512 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231751ff12a4b1b41fcae8b65e8c488c4be to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231751ff12a4b1b41fcae8b65e8c488c4be 2023-02-09 14:55:59,513 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:55:59,518 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/3ff2603f983a478ebea9ab51ac638d75 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/3ff2603f983a478ebea9ab51ac638d75 2023-02-09 14:55:59,522 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 4a620fe72a6269f78f560319e3947df4/family1 of 4a620fe72a6269f78f560319e3947df4 into 3ff2603f983a478ebea9ab51ac638d75(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:59,522 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:59,522 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675954558625; duration=0sec 2023-02-09 14:55:59,522 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:59,529 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:59,530 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:59,538 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/6f2fc784145c4af4bb89e55c501f9ce3 first=Optional[r1] last=Optional[r4] 2023-02-09 14:55:59,538 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/6f2fc784145c4af4bb89e55c501f9ce3}] 2023-02-09 14:55:59,541 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/6f2fc784145c4af4bb89e55c501f9ce3 for inclusion in 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:59,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 14:55:59,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:55:59,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:55:59,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/6f2fc784145c4af4bb89e55c501f9ce3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__foujoknnv799ta7dsj0sfbvff1osm831s2fo3f4lsa0p8sb04euiom12e2s1489l/family1/6f2fc784145c4af4bb89e55c501f9ce3 2023-02-09 14:55:59,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__foujoknnv799ta7dsj0sfbvff1osm831s2fo3f4lsa0p8sb04euiom12e2s1489l/family1/6f2fc784145c4af4bb89e55c501f9ce3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_ 2023-02-09 14:55:59,546 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__foujoknnv799ta7dsj0sfbvff1osm831s2fo3f4lsa0p8sb04euiom12e2s1489l/family1/6f2fc784145c4af4bb89e55c501f9ce3 into 4a620fe72a6269f78f560319e3947df4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_ - updating store file list. 2023-02-09 14:55:59,549 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_ into 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:55:59,549 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__foujoknnv799ta7dsj0sfbvff1osm831s2fo3f4lsa0p8sb04euiom12e2s1489l/family1/6f2fc784145c4af4bb89e55c501f9ce3 into 4a620fe72a6269f78f560319e3947df4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_) 2023-02-09 14:55:59,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__foujoknnv799ta7dsj0sfbvff1osm831s2fo3f4lsa0p8sb04euiom12e2s1489l/family1/6f2fc784145c4af4bb89e55c501f9ce3 2023-02-09 14:55:59,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 14:55:59,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 14:55:59,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(1912): 4a620fe72a6269f78f560319e3947df4/family1 is initiating minor compaction (all files) 2023-02-09 14:55:59,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1caf85f6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:59,550 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4a620fe72a6269f78f560319e3947df4/family1 in testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:55:59,551 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/3ff2603f983a478ebea9ab51ac638d75, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp, totalSize=178.1 K 2023-02-09 14:55:59,551 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting e33605916e8249d4989182d3a9a4f8b0, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840463 2023-02-09 14:55:59,551 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 3ff2603f983a478ebea9ab51ac638d75, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240463 2023-02-09 14:55:59,552 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] compactions.Compactor(207): Compacting 8c3db54a6d78483094e294e48096043a_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240463 2023-02-09 14:55:59,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:55:59,553 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:59,554 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201feb58f9214204ea7808253b6430861d8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201feb58f9214204ea7808253b6430861d8 2023-02-09 14:55:59,555 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512052ce79179a35840b7b100f8b998dc2b3d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512052ce79179a35840b7b100f8b998dc2b3d 2023-02-09 14:55:59,556 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122826e231f99d45459a9bef90026c4ceae6 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122826e231f99d45459a9bef90026c4ceae6 2023-02-09 14:55:59,557 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316232e6ec0ade40238dc01e4059796386 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316232e6ec0ade40238dc01e4059796386 2023-02-09 14:55:59,557 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 14:55:59,557 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 14:55:59,558 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:55:59,558 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:55:59,582 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4a620fe72a6269f78f560319e3947df4#family1#compaction#166 average throughput is 11.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:59,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_] to archive 2023-02-09 14:55:59,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:59,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e1aa30069b694a7484edba5e3c9e2807 2023-02-09 14:55:59,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/be64605936f64a71aaac547f2b270eb8 2023-02-09 14:55:59,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7dd0879e0dff4bd99b7c577741343c29 2023-02-09 14:55:59,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/d7e39efe925943d5862df93a58c8cb06 2023-02-09 14:55:59,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dd5a46f430604f06bda85e45c316099e 2023-02-09 14:55:59,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/616ff826959a4262a04183bb8bc2e108 2023-02-09 14:55:59,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/7519da7a8cd64c4a98a151f45df7e1b1 2023-02-09 14:55:59,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4106f99dee834f849c25857fca0387d5 2023-02-09 14:55:59,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/f42b1fed00ae46309847737cdcc6b7e8 2023-02-09 14:55:59,634 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/63226b643b404ded95ab5bf4ff4cd786_SeqId_44_ 2023-02-09 14:56:00,412 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e8a0acc10c824294b0c7607321042812 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e8a0acc10c824294b0c7607321042812 2023-02-09 14:56:00,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:56:00,429 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:00,429 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:00,436 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/42fd079a9d4845d3bcf9754b232ac70d first=Optional[r00] last=Optional[r8] 2023-02-09 14:56:00,437 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/42fd079a9d4845d3bcf9754b232ac70d}] 2023-02-09 14:56:00,439 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/42fd079a9d4845d3bcf9754b232ac70d for inclusion in 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:56:00,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 14:56:00,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 14:56:00,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:56:00,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/42fd079a9d4845d3bcf9754b232ac70d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5agqljp7fhrb1r9lh7lb4lp05mp12pigujhrf0sgpsbmej6udnm5n8tpt4secjf3/family1/42fd079a9d4845d3bcf9754b232ac70d 2023-02-09 14:56:00,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5agqljp7fhrb1r9lh7lb4lp05mp12pigujhrf0sgpsbmej6udnm5n8tpt4secjf3/family1/42fd079a9d4845d3bcf9754b232ac70d as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4d52f0301c9a4c9f98616b8df9c167d3_SeqId_49_ 2023-02-09 14:56:00,444 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5agqljp7fhrb1r9lh7lb4lp05mp12pigujhrf0sgpsbmej6udnm5n8tpt4secjf3/family1/42fd079a9d4845d3bcf9754b232ac70d into 4a620fe72a6269f78f560319e3947df4/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4d52f0301c9a4c9f98616b8df9c167d3_SeqId_49_ - updating store file list. 2023-02-09 14:56:00,446 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4d52f0301c9a4c9f98616b8df9c167d3_SeqId_49_ into 4a620fe72a6269f78f560319e3947df4/family1 2023-02-09 14:56:00,446 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5agqljp7fhrb1r9lh7lb4lp05mp12pigujhrf0sgpsbmej6udnm5n8tpt4secjf3/family1/42fd079a9d4845d3bcf9754b232ac70d into 4a620fe72a6269f78f560319e3947df4/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/4d52f0301c9a4c9f98616b8df9c167d3_SeqId_49_) 2023-02-09 14:56:00,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMinorCompactionWithMonthlyPolicy__5agqljp7fhrb1r9lh7lb4lp05mp12pigujhrf0sgpsbmej6udnm5n8tpt4secjf3/family1/42fd079a9d4845d3bcf9754b232ac70d 2023-02-09 14:56:00,448 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:56:00,449 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:00,450 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112041a96f5d52c34578ae823dfb246a99a0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112041a96f5d52c34578ae823dfb246a99a0 2023-02-09 14:56:00,451 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511287b2b1a10098f472db5b90d2ef7e40415 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511287b2b1a10098f472db5b90d2ef7e40415 2023-02-09 14:56:00,452 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130618dc89e35e94875b98b83533b70ea0b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130618dc89e35e94875b98b83533b70ea0b 2023-02-09 14:56:00,452 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 14:56:00,452 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 14:56:00,452 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:56:00,452 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:56:00,452 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 14:56:00,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 14:56:00,452 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:56:00,454 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 14:56:00,455 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.3680 sec 2023-02-09 14:56:00,652 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=259 (was 255) - Thread LEAK? -, OpenFileDescriptor=664 (was 649) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=81 (was 86), ProcessCount=168 (was 168), AvailableMemoryMB=3031 (was 2941) - AvailableMemoryMB LEAK? - 2023-02-09 14:56:00,664 INFO [Listener at localhost/33189] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=259, OpenFileDescriptor=664, MaxFileDescriptor=60000, SystemLoadAverage=81, ProcessCount=168, AvailableMemoryMB=3031 2023-02-09 14:56:00,683 DEBUG [Listener at localhost/33189] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-09 14:56:00,705 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:00,706 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:00,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 14:56:00,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-09 14:56:00,741 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:56:00,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-09 14:56:00,741 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:00,741 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:00,741 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:56:00,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:00,743 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:00,743 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:00,743 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06 empty. 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc empty. 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb empty. 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:00,744 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:00,744 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-09 14:56:00,817 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/.tmp/family1/dc1adbeb8f4e48aebfd90bd11cf6dad2 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/dc1adbeb8f4e48aebfd90bd11cf6dad2 2023-02-09 14:56:00,821 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 4a620fe72a6269f78f560319e3947df4/family1 of 4a620fe72a6269f78f560319e3947df4 into dc1adbeb8f4e48aebfd90bd11cf6dad2(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:56:00,821 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4a620fe72a6269f78f560319e3947df4: 2023-02-09 14:56:00,821 INFO [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675954559550; duration=1sec 2023-02-09 14:56:00,821 DEBUG [RS:0;jenkins-hbase4:43455-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:56:00,842 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:00,842 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:00,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:00,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/3ff2603f983a478ebea9ab51ac638d75, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_] to archive 2023-02-09 14:56:00,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:00,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/e33605916e8249d4989182d3a9a4f8b0 2023-02-09 14:56:00,924 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/3ff2603f983a478ebea9ab51ac638d75 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/3ff2603f983a478ebea9ab51ac638d75 2023-02-09 14:56:00,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_ to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/family1/8c3db54a6d78483094e294e48096043a_SeqId_47_ 2023-02-09 14:56:01,043 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:01,043 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:01,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:01,151 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-09 14:56:01,153 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => 932a93e1be1254f02a4c55d7755b4abb, NAME => 'testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:56:01,153 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => d22c6e5399984f7dc80f2f448d9fffcc, NAME => 'testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:56:01,153 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 323cead325f2feb2545ff228bca36c06, NAME => 'testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/.tmp 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 323cead325f2feb2545ff228bca36c06, disabling compactions & flushes 2023-02-09 14:56:01,163 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. after waiting 0 ms 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,163 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,163 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 323cead325f2feb2545ff228bca36c06: Waiting for close lock at 1675954561163Disabling compacts and flushes for region at 1675954561163Disabling writes for close at 1675954561163Writing region close event to WAL at 1675954561163Closed at 1675954561163 2023-02-09 14:56:01,345 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:01,345 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:01,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing d22c6e5399984f7dc80f2f448d9fffcc, disabling compactions & flushes 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing 932a93e1be1254f02a4c55d7755b4abb, disabling compactions & flushes 2023-02-09 14:56:01,563 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,563 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. after waiting 0 ms 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. after waiting 0 ms 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,563 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,563 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,563 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for d22c6e5399984f7dc80f2f448d9fffcc: Waiting for close lock at 1675954561563Disabling compacts and flushes for region at 1675954561563Disabling writes for close at 1675954561563Writing region close event to WAL at 1675954561563Closed at 1675954561563 2023-02-09 14:56:01,564 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for 932a93e1be1254f02a4c55d7755b4abb: Waiting for close lock at 1675954561563Disabling compacts and flushes for region at 1675954561563Disabling writes for close at 1675954561563Writing region close event to WAL at 1675954561563Closed at 1675954561563 2023-02-09 14:56:01,565 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:56:01,566 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954561566"}]},"ts":"1675954561566"} 2023-02-09 14:56:01,566 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954561566"}]},"ts":"1675954561566"} 2023-02-09 14:56:01,566 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954561566"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954561566"}]},"ts":"1675954561566"} 2023-02-09 14:56:01,568 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 14:56:01,569 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:56:01,569 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954561569"}]},"ts":"1675954561569"} 2023-02-09 14:56:01,570 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-09 14:56:01,572 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=323cead325f2feb2545ff228bca36c06, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=932a93e1be1254f02a4c55d7755b4abb, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d22c6e5399984f7dc80f2f448d9fffcc, ASSIGN}] 2023-02-09 14:56:01,574 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d22c6e5399984f7dc80f2f448d9fffcc, ASSIGN 2023-02-09 14:56:01,574 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=323cead325f2feb2545ff228bca36c06, ASSIGN 2023-02-09 14:56:01,574 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=932a93e1be1254f02a4c55d7755b4abb, ASSIGN 2023-02-09 14:56:01,574 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d22c6e5399984f7dc80f2f448d9fffcc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:56:01,575 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=323cead325f2feb2545ff228bca36c06, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:56:01,575 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=932a93e1be1254f02a4c55d7755b4abb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43455,1675954416872; forceNewPlan=false, retain=false 2023-02-09 14:56:01,726 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=932a93e1be1254f02a4c55d7755b4abb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,726 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=323cead325f2feb2545ff228bca36c06, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,726 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954561726"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954561726"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954561726"}]},"ts":"1675954561726"} 2023-02-09 14:56:01,726 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=d22c6e5399984f7dc80f2f448d9fffcc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,726 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561726"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954561726"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954561726"}]},"ts":"1675954561726"} 2023-02-09 14:56:01,727 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561726"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954561726"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954561726"}]},"ts":"1675954561726"} 2023-02-09 14:56:01,728 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure 932a93e1be1254f02a4c55d7755b4abb, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:56:01,728 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 323cead325f2feb2545ff228bca36c06, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:56:01,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure d22c6e5399984f7dc80f2f448d9fffcc, server=jenkins-hbase4.apache.org,43455,1675954416872}] 2023-02-09 14:56:01,846 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:01,846 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:01,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:01,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 932a93e1be1254f02a4c55d7755b4abb, NAME => 'testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 14:56:01,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,905 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,906 DEBUG [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1 2023-02-09 14:56:01,906 DEBUG [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1 2023-02-09 14:56:01,907 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 932a93e1be1254f02a4c55d7755b4abb columnFamilyName family1 2023-02-09 14:56:01,908 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] regionserver.HStore(310): Store=932a93e1be1254f02a4c55d7755b4abb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,908 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,909 DEBUG [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2 2023-02-09 14:56:01,909 DEBUG [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2 2023-02-09 14:56:01,909 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 932a93e1be1254f02a4c55d7755b4abb columnFamilyName family2 2023-02-09 14:56:01,910 INFO [StoreOpener-932a93e1be1254f02a4c55d7755b4abb-1] regionserver.HStore(310): Store=932a93e1be1254f02a4c55d7755b4abb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:56:01,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:01,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:56:01,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 932a93e1be1254f02a4c55d7755b4abb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64608410, jitterRate=-0.03725966811180115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:56:01,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 932a93e1be1254f02a4c55d7755b4abb: Running coprocessor pre-open hook at 1675954561883Writing region info on filesystem at 1675954561883Initializing all the Stores at 1675954561883Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561883Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561884 (+1 ms)Cleaning up temporary data from old regions at 1675954561912 (+28 ms)Cleaning up detritus from prior splits at 1675954561912Running coprocessor post-open hooks at 1675954561916 (+4 ms)Region opened successfully at 1675954561916 2023-02-09 14:56:01,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb., pid=120, masterSystemTime=1675954561880 2023-02-09 14:56:01,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:01,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d22c6e5399984f7dc80f2f448d9fffcc, NAME => 'testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 14:56:01,918 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=932a93e1be1254f02a4c55d7755b4abb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,918 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954561918"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954561918"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954561918"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954561918"}]},"ts":"1675954561918"} 2023-02-09 14:56:01,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-02-09 14:56:01,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure 932a93e1be1254f02a4c55d7755b4abb, server=jenkins-hbase4.apache.org,43455,1675954416872 in 191 msec 2023-02-09 14:56:01,921 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=932a93e1be1254f02a4c55d7755b4abb, ASSIGN in 348 msec 2023-02-09 14:56:01,939 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,941 DEBUG [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1 2023-02-09 14:56:01,941 DEBUG [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1 2023-02-09 14:56:01,941 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d22c6e5399984f7dc80f2f448d9fffcc columnFamilyName family1 2023-02-09 14:56:01,942 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] regionserver.HStore(310): Store=d22c6e5399984f7dc80f2f448d9fffcc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,942 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,943 DEBUG [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2 2023-02-09 14:56:01,943 DEBUG [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2 2023-02-09 14:56:01,943 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d22c6e5399984f7dc80f2f448d9fffcc columnFamilyName family2 2023-02-09 14:56:01,944 INFO [StoreOpener-d22c6e5399984f7dc80f2f448d9fffcc-1] regionserver.HStore(310): Store=d22c6e5399984f7dc80f2f448d9fffcc/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:56:01,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:01,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:56:01,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d22c6e5399984f7dc80f2f448d9fffcc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70151883, jitterRate=0.0453445166349411}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:56:01,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d22c6e5399984f7dc80f2f448d9fffcc: Running coprocessor pre-open hook at 1675954561918Writing region info on filesystem at 1675954561918Initializing all the Stores at 1675954561919 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561919Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561919Cleaning up temporary data from old regions at 1675954561945 (+26 ms)Cleaning up detritus from prior splits at 1675954561946 (+1 ms)Running coprocessor post-open hooks at 1675954561949 (+3 ms)Region opened successfully at 1675954561949 2023-02-09 14:56:01,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc., pid=122, masterSystemTime=1675954561880 2023-02-09 14:56:01,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:01,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 323cead325f2feb2545ff228bca36c06, NAME => 'testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 14:56:01,951 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=d22c6e5399984f7dc80f2f448d9fffcc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:56:01,951 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561951"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954561951"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954561951"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954561951"}]},"ts":"1675954561951"} 2023-02-09 14:56:01,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-02-09 14:56:01,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure d22c6e5399984f7dc80f2f448d9fffcc, server=jenkins-hbase4.apache.org,43455,1675954416872 in 222 msec 2023-02-09 14:56:01,954 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d22c6e5399984f7dc80f2f448d9fffcc, ASSIGN in 381 msec 2023-02-09 14:56:01,976 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,977 DEBUG [StoreOpener-323cead325f2feb2545ff228bca36c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1 2023-02-09 14:56:01,977 DEBUG [StoreOpener-323cead325f2feb2545ff228bca36c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1 2023-02-09 14:56:01,978 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 323cead325f2feb2545ff228bca36c06 columnFamilyName family1 2023-02-09 14:56:01,978 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] regionserver.HStore(310): Store=323cead325f2feb2545ff228bca36c06/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,979 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,980 DEBUG [StoreOpener-323cead325f2feb2545ff228bca36c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2 2023-02-09 14:56:01,980 DEBUG [StoreOpener-323cead325f2feb2545ff228bca36c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2 2023-02-09 14:56:01,980 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 323cead325f2feb2545ff228bca36c06 columnFamilyName family2 2023-02-09 14:56:01,980 INFO [StoreOpener-323cead325f2feb2545ff228bca36c06-1] regionserver.HStore(310): Store=323cead325f2feb2545ff228bca36c06/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:56:01,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:56:01,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:01,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:56:01,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 323cead325f2feb2545ff228bca36c06; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67272784, jitterRate=0.002442598342895508}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:56:01,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 323cead325f2feb2545ff228bca36c06: Running coprocessor pre-open hook at 1675954561951Writing region info on filesystem at 1675954561951Initializing all the Stores at 1675954561952 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561952Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675954561952Cleaning up temporary data from old regions at 1675954561982 (+30 ms)Cleaning up detritus from prior splits at 1675954561982Running coprocessor post-open hooks at 1675954561985 (+3 ms)Region opened successfully at 1675954561985 2023-02-09 14:56:01,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06., pid=121, masterSystemTime=1675954561880 2023-02-09 14:56:01,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:01,987 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=323cead325f2feb2545ff228bca36c06, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:01,987 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675954561987"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954561987"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954561987"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954561987"}]},"ts":"1675954561987"} 2023-02-09 14:56:01,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-02-09 14:56:01,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 323cead325f2feb2545ff228bca36c06, server=jenkins-hbase4.apache.org,43455,1675954416872 in 260 msec 2023-02-09 14:56:01,991 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-09 14:56:01,991 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=323cead325f2feb2545ff228bca36c06, ASSIGN in 417 msec 2023-02-09 14:56:01,991 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:56:01,991 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954561991"}]},"ts":"1675954561991"} 2023-02-09 14:56:01,992 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-09 14:56:01,994 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:56:01,995 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2560 sec 2023-02-09 14:56:02,819 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-09 14:56:02,847 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:02,847 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:02,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 14:56:02,848 INFO [Listener at localhost/33189] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-09 14:56:02,854 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:02,854 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:02,854 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:02,856 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:02,856 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:02,857 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:56:02,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 14:56:02,858 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 14:56:02,858 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:02,858 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 14:56:02,858 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 14:56:02,859 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:02,859 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:02,861 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:02,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:02,861 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:02,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:02,861 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:02,861 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:56:02,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:02,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:56:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:02,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 14:56:02,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 14:56:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 14:56:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:02,862 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 14:56:02,862 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:02,862 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:02,862 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. started... 2023-02-09 14:56:02,862 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. started... 2023-02-09 14:56:02,863 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:56:02,864 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:56:02,864 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 323cead325f2feb2545ff228bca36c06 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:56:02,864 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d22c6e5399984f7dc80f2f448d9fffcc 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:56:02,864 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. started... 2023-02-09 14:56:02,865 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 932a93e1be1254f02a4c55d7755b4abb 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 14:56:02,934 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ee7f3a55e10d4c37a2cbfd24737e356f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ee7f3a55e10d4c37a2cbfd24737e356f 2023-02-09 14:56:03,322 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209272fbc0391f84973bd6b4075e09ba731 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209272fbc0391f84973bd6b4075e09ba731 2023-02-09 14:56:03,324 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209644da45b9399457f8245612b9b60b6fa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209644da45b9399457f8245612b9b60b6fa 2023-02-09 14:56:03,350 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/f67ec26153b24ae4aa08b859e206841c 2023-02-09 14:56:03,737 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/08de665b63de4af1bddac1be722bdb07 2023-02-09 14:56:03,738 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/0110ffd9694646c696adce49533864dc 2023-02-09 14:56:03,783 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020936e8ca711a57404fb89cab44e99bb465 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020936e8ca711a57404fb89cab44e99bb465 2023-02-09 14:56:04,171 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f794f1b181f4472a87d7fccbf095d4d0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209f794f1b181f4472a87d7fccbf095d4d0 2023-02-09 14:56:04,183 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209abf710b219844c51a5c7a19da0c325e1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230209abf710b219844c51a5c7a19da0c325e1 2023-02-09 14:56:04,187 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/57ada83452454b8ab55dc1ef7faa160a 2023-02-09 14:56:04,190 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/f67ec26153b24ae4aa08b859e206841c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c 2023-02-09 14:56:04,214 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 14:56:04,215 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/57ada83452454b8ab55dc1ef7faa160a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a 2023-02-09 14:56:04,218 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:56:04,218 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 932a93e1be1254f02a4c55d7755b4abb in 1353ms, sequenceid=5, compaction requested=false 2023-02-09 14:56:04,219 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:04,219 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:04,219 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:56:04,576 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/7d6b7db223424a46b85f7c63f9658505 2023-02-09 14:56:04,580 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/08de665b63de4af1bddac1be722bdb07 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07 2023-02-09 14:56:04,587 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/585f28e836ad4926af3a560b2d63c403 2023-02-09 14:56:04,591 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/0110ffd9694646c696adce49533864dc as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc 2023-02-09 14:56:04,605 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 14:56:04,606 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/7d6b7db223424a46b85f7c63f9658505 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505 2023-02-09 14:56:04,609 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:56:04,610 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d22c6e5399984f7dc80f2f448d9fffcc in 1746ms, sequenceid=5, compaction requested=false 2023-02-09 14:56:04,610 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:04,610 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:04,610 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:56:04,614 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 14:56:04,614 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/585f28e836ad4926af3a560b2d63c403 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403 2023-02-09 14:56:04,618 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 14:56:04,618 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 323cead325f2feb2545ff228bca36c06 in 1754ms, sequenceid=5, compaction requested=false 2023-02-09 14:56:04,619 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:04,619 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:04,619 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:56:04,619 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:56:04,619 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,619 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 14:56:04,619 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 14:56:04,624 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,624 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:04,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:04,624 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,625 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:56:04,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:04,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:04,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:04,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 14:56:04,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70b496e4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:56:04,626 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 14:56:04,626 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,627 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,627 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 14:56:04,627 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 14:56:04,627 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:56:04,627 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,627 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:56:04,628 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 14:56:04,628 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:04,628 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,628 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 14:56:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:04,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:04,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:56:04,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 14:56:04,630 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 14:56:04,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:56:04,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:56:04,630 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 14:56:04,630 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:56:04,631 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,631 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,631 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:04,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:04,632 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,633 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:04,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:04,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:04,636 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:04,636 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 14:56:04,636 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:56:04,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 14:56:04,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:04,637 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:56:04,636 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,737 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:56:04,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:04,737 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:04,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:56:04,739 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:04,739 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:04,739 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:04,741 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:04,741 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:04,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:56:04,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 14:56:04,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 14:56:04,743 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 14:56:04,743 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:04,743 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 14:56:04,743 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 14:56:04,743 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,743 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:04,745 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:04,745 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:04,745 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:04,745 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:56:04,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:04,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:04,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:56:04,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:04,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 14:56:04,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 14:56:04,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 14:56:04,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 14:56:04,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:04,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:04,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:56:04,746 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. started... 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. started... 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:04,746 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. started... 2023-02-09 14:56:04,747 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d22c6e5399984f7dc80f2f448d9fffcc 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:56:04,747 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 932a93e1be1254f02a4c55d7755b4abb 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:56:04,747 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 323cead325f2feb2545ff228bca36c06 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 14:56:05,196 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302096bf6829f1ea944a9a6f2c30715aecacb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302096bf6829f1ea944a9a6f2c30715aecacb 2023-02-09 14:56:05,196 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302091b6832b23d11447e9e1cc82335a8b7fe to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302091b6832b23d11447e9e1cc82335a8b7fe 2023-02-09 14:56:05,196 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302095258d8e775384c90921910c37fbaa28e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095258d8e775384c90921910c37fbaa28e 2023-02-09 14:56:05,612 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/45a1150e467f430cab93d6fc89ab6f0f 2023-02-09 14:56:05,612 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/4cf7089b868646a996e2ced78cbab76b 2023-02-09 14:56:05,612 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/33a7d0c9b74448e5a7766d6004843198 2023-02-09 14:56:05,674 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302090e230b67a4c94c9098edd6d5f44914a2 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302090e230b67a4c94c9098edd6d5f44914a2 2023-02-09 14:56:05,674 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209cd9764fd5cce4257b1a60f64f048acd4 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd9764fd5cce4257b1a60f64f048acd4 2023-02-09 14:56:06,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020948078b6fc5804d04a61741db6982df23 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020948078b6fc5804d04a61741db6982df23 2023-02-09 14:56:06,085 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/0085bbb1f6d340c5b738ea1aa0ce8f83 2023-02-09 14:56:06,085 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/dcd0c61b06be492d82975a0865884a45 2023-02-09 14:56:06,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/4cf7089b868646a996e2ced78cbab76b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b 2023-02-09 14:56:06,088 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/33a7d0c9b74448e5a7766d6004843198 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198 2023-02-09 14:56:06,113 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 14:56:06,113 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 14:56:06,114 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/dcd0c61b06be492d82975a0865884a45 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45 2023-02-09 14:56:06,114 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/0085bbb1f6d340c5b738ea1aa0ce8f83 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83 2023-02-09 14:56:06,117 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:56:06,118 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:56:06,118 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 323cead325f2feb2545ff228bca36c06 in 1371ms, sequenceid=9, compaction requested=false 2023-02-09 14:56:06,118 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:06,118 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:06,118 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:56:06,118 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d22c6e5399984f7dc80f2f448d9fffcc in 1372ms, sequenceid=9, compaction requested=false 2023-02-09 14:56:06,118 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:06,118 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:06,118 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:56:06,479 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/0ecda03151b543c9bd64dd2fdff8ce58 2023-02-09 14:56:06,483 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/45a1150e467f430cab93d6fc89ab6f0f as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f 2023-02-09 14:56:06,508 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 14:56:06,509 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/0ecda03151b543c9bd64dd2fdff8ce58 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58 2023-02-09 14:56:06,512 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 14:56:06,513 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 932a93e1be1254f02a4c55d7755b4abb in 1767ms, sequenceid=9, compaction requested=false 2023-02-09 14:56:06,513 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:06,513 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:56:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:56:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 14:56:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 14:56:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,515 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:06,515 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:56:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:06,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 14:56:06,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13a46cb9[Count = 0] remaining members to acquire global barrier 2023-02-09 14:56:06,517 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 14:56:06,517 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,518 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 14:56:06,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 14:56:06,518 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:56:06,518 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:56:06,521 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,521 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 14:56:06,521 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:06,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:06,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:06,521 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 14:56:06,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:06,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:06,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:56:06,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 14:56:06,523 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 14:56:06,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:56:06,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:56:06,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 14:56:06,523 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:56:06,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,524 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,525 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:06,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:06,526 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:06,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:06,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:06,528 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:06,528 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 14:56:06,528 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:56:06,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:06,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 14:56:06,528 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,529 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:56:06,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 14:56:06,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:06,629 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:56:06,629 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:06,629 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:06,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:56:06,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:06,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:06,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:06,633 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:06,633 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:06,633 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:56:06,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 14:56:06,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 14:56:06,635 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 14:56:06,635 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:06,635 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 14:56:06,635 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 14:56:06,636 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,636 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:06,637 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:06,637 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:06,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:06,637 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:06,637 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:56:06,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:06,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:06,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:56:06,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:06,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 14:56:06,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 14:56:06,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 14:56:06,638 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 14:56:06,638 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:06,638 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. started... 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. started... 2023-02-09 14:56:06,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d22c6e5399984f7dc80f2f448d9fffcc 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:56:06,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 323cead325f2feb2545ff228bca36c06 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:56:06,639 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:56:06,639 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:06,639 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. started... 2023-02-09 14:56:06,639 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 932a93e1be1254f02a4c55d7755b4abb 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 14:56:06,696 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302093a72bbc8d156440ab221a4d8cb7630c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302093a72bbc8d156440ab221a4d8cb7630c0 2023-02-09 14:56:07,094 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209340d30c6a51b450999733056cd8aed68 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209340d30c6a51b450999733056cd8aed68 2023-02-09 14:56:07,097 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209eb760f79302040dba2ca05773ae19c3f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209eb760f79302040dba2ca05773ae19c3f 2023-02-09 14:56:07,111 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/bbfa873286414797a1d2f81d76e9bd55 2023-02-09 14:56:07,421 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:56:07,518 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/9c9fbdc22de042e49d20a2f201eb2f4a 2023-02-09 14:56:07,521 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/c12279a570424da1a035655fef9269f8 2023-02-09 14:56:07,553 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ecd534ad02104cadb9a491d7f37ce9d8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209ecd534ad02104cadb9a491d7f37ce9d8 2023-02-09 14:56:07,957 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302093b490183b00f4fe6baf65971b49c8e29 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302093b490183b00f4fe6baf65971b49c8e29 2023-02-09 14:56:07,958 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209adfe843478a548bf89f8fa836ad36f77 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209adfe843478a548bf89f8fa836ad36f77 2023-02-09 14:56:07,960 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/e2bd84fc264c410eb41326c3522047cd 2023-02-09 14:56:07,963 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/58382c35f68d4fa2afb96cdcad2d46bd 2023-02-09 14:56:07,965 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/bbfa873286414797a1d2f81d76e9bd55 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55 2023-02-09 14:56:07,968 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/9c9fbdc22de042e49d20a2f201eb2f4a as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a 2023-02-09 14:56:07,998 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 14:56:07,999 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/58382c35f68d4fa2afb96cdcad2d46bd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd 2023-02-09 14:56:08,001 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 14:56:08,002 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/e2bd84fc264c410eb41326c3522047cd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd 2023-02-09 14:56:08,003 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:56:08,004 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d22c6e5399984f7dc80f2f448d9fffcc in 1365ms, sequenceid=13, compaction requested=true 2023-02-09 14:56:08,004 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:08,004 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:08,004 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:56:08,006 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:56:08,006 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 932a93e1be1254f02a4c55d7755b4abb in 1367ms, sequenceid=13, compaction requested=true 2023-02-09 14:56:08,006 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:08,007 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:08,007 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:56:08,362 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/ff861395572d4cb1a85ace8b2e9aaeda 2023-02-09 14:56:08,366 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/c12279a570424da1a035655fef9269f8 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8 2023-02-09 14:56:08,397 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 14:56:08,398 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/ff861395572d4cb1a85ace8b2e9aaeda as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda 2023-02-09 14:56:08,401 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 14:56:08,402 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 323cead325f2feb2545ff228bca36c06 in 1763ms, sequenceid=13, compaction requested=true 2023-02-09 14:56:08,402 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:08,402 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:56:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:56:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 14:56:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 14:56:08,405 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,405 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:08,405 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:08,405 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:08,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:08,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 14:56:08,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6193501[Count = 0] remaining members to acquire global barrier 2023-02-09 14:56:08,406 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 14:56:08,406 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,407 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,407 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 14:56:08,407 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 14:56:08,407 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:56:08,407 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,407 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:56:08,409 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 14:56:08,409 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,409 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,409 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 14:56:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:56:08,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 14:56:08,411 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 14:56:08,411 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:56:08,411 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:56:08,411 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 14:56:08,411 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:56:08,412 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,412 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,412 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:08,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,413 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:08,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:08,414 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:08,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,416 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,416 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:08,416 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:08,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 14:56:08,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:08,417 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:08,417 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:08,417 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,517 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:56:08,517 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:08,517 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:08,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:56:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:08,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 14:56:08,521 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:08,521 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:08,522 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:56:08,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 14:56:08,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 14:56:08,523 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 14:56:08,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:08,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 14:56:08,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 14:56:08,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,524 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:08,527 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:08,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:08,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:08,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:08,527 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:56:08,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:08,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:08,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:56:08,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:08,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 14:56:08,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 14:56:08,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 14:56:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 14:56:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. started... 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. started... 2023-02-09 14:56:08,529 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. started... 2023-02-09 14:56:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:56:08,529 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d22c6e5399984f7dc80f2f448d9fffcc 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:56:08,529 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:56:08,529 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 323cead325f2feb2545ff228bca36c06 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:56:08,529 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 932a93e1be1254f02a4c55d7755b4abb 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 14:56:08,980 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ecd01cd61ec443f19500d621ae3e8e21 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ecd01cd61ec443f19500d621ae3e8e21 2023-02-09 14:56:08,981 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209ff3305da875d41aab125be7a31171d2d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209ff3305da875d41aab125be7a31171d2d 2023-02-09 14:56:08,989 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209fd5b70b485424570b91baf01c6231c74 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209fd5b70b485424570b91baf01c6231c74 2023-02-09 14:56:09,004 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/e391328208da4cc09de5aecd185ea231 2023-02-09 14:56:09,396 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3 2023-02-09 14:56:09,398 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/1c3610fce6014c94991dce63294d747b 2023-02-09 14:56:09,436 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209e6c08305e1ee473d8b63324fe3a5a539 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209e6c08305e1ee473d8b63324fe3a5a539 2023-02-09 14:56:09,442 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302098752542804964a45806742cb4cfd7395 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302098752542804964a45806742cb4cfd7395 2023-02-09 14:56:09,842 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302092129ebbf94a64f0c8b19966cdd6958b7 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302092129ebbf94a64f0c8b19966cdd6958b7 2023-02-09 14:56:09,848 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/ea7bd90e35d34544821f93c18de7ff72 2023-02-09 14:56:09,848 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/cf52d117773241d784a30277d877a074 2023-02-09 14:56:09,852 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/e391328208da4cc09de5aecd185ea231 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231 2023-02-09 14:56:09,852 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/1c3610fce6014c94991dce63294d747b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b 2023-02-09 14:56:09,876 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 14:56:09,877 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/ea7bd90e35d34544821f93c18de7ff72 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72 2023-02-09 14:56:09,881 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:56:09,882 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 932a93e1be1254f02a4c55d7755b4abb in 1353ms, sequenceid=17, compaction requested=true 2023-02-09 14:56:09,883 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 14:56:09,883 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:09,883 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:09,883 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:56:09,883 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/cf52d117773241d784a30277d877a074 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074 2023-02-09 14:56:09,888 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:56:09,888 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d22c6e5399984f7dc80f2f448d9fffcc in 1359ms, sequenceid=17, compaction requested=true 2023-02-09 14:56:09,889 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:09,889 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:09,889 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:56:10,248 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/6f244e63e5fe45a8aa7ed2709c8a9277 2023-02-09 14:56:10,252 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3 2023-02-09 14:56:10,281 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 14:56:10,282 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/6f244e63e5fe45a8aa7ed2709c8a9277 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277 2023-02-09 14:56:10,285 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 14:56:10,286 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 323cead325f2feb2545ff228bca36c06 in 1757ms, sequenceid=17, compaction requested=true 2023-02-09 14:56:10,286 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:10,286 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:10,286 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:56:10,286 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:56:10,286 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 14:56:10,287 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 14:56:10,288 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,288 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:10,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:10,288 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,288 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:56:10,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:10,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:10,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:10,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 14:56:10,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1951d28e[Count = 0] remaining members to acquire global barrier 2023-02-09 14:56:10,290 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 14:56:10,290 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,291 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 14:56:10,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 14:56:10,291 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:56:10,291 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,291 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:56:10,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 14:56:10,292 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,292 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 14:56:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:10,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:56:10,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 14:56:10,295 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 14:56:10,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:56:10,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:56:10,295 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 14:56:10,295 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:56:10,298 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,298 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:10,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:10,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,298 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:10,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:10,299 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:10,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:10,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:10,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,300 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:10,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:10,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:10,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:10,302 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:10,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 14:56:10,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:10,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:10,302 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,302 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 14:56:10,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:10,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,303 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:56:10,403 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:56:10,404 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:10,404 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:10,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:56:10,412 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:10,419 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:10,428 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:10,429 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:10,520 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:10,520 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:10,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:56:10,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 14:56:10,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 14:56:10,523 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 14:56:10,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:10,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 14:56:10,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 14:56:10,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,523 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:10,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,525 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:10,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:10,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:10,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:10,525 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:56:10,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:10,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:10,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 14:56:10,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 14:56:10,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 14:56:10,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 14:56:10,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:56:10,526 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. started... 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. started... 2023-02-09 14:56:10,527 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d22c6e5399984f7dc80f2f448d9fffcc 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:56:10,527 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 323cead325f2feb2545ff228bca36c06 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:56:10,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 14:56:10,527 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:10,527 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. started... 2023-02-09 14:56:10,528 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 932a93e1be1254f02a4c55d7755b4abb 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 14:56:10,982 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/3916ed0b8d074fc3b2d565f63181cd28 2023-02-09 14:56:10,982 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/76dd31bac2d74e73aa0cb36cfbdb746b 2023-02-09 14:56:11,007 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76dd31bac2d74e73aa0cb36cfbdb746b 2023-02-09 14:56:11,013 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3916ed0b8d074fc3b2d565f63181cd28 2023-02-09 14:56:11,036 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/a9e39d731dd145dd8b909e8823d473e1 2023-02-09 14:56:11,040 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9e39d731dd145dd8b909e8823d473e1 2023-02-09 14:56:11,040 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/3916ed0b8d074fc3b2d565f63181cd28 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28 2023-02-09 14:56:11,070 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3916ed0b8d074fc3b2d565f63181cd28 2023-02-09 14:56:11,070 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 14:56:11,071 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/a9e39d731dd145dd8b909e8823d473e1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1 2023-02-09 14:56:11,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9e39d731dd145dd8b909e8823d473e1 2023-02-09 14:56:11,074 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:56:11,075 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 323cead325f2feb2545ff228bca36c06 in 548ms, sequenceid=23, compaction requested=true 2023-02-09 14:56:11,075 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:11,075 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:11,075 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 14:56:11,378 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/17aa5eaa1ea644fc90f46b655ccedbba 2023-02-09 14:56:11,403 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17aa5eaa1ea644fc90f46b655ccedbba 2023-02-09 14:56:11,827 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/692a8c7a466a4f28991fcee05ea8529e 2023-02-09 14:56:11,830 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 692a8c7a466a4f28991fcee05ea8529e 2023-02-09 14:56:11,830 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/76dd31bac2d74e73aa0cb36cfbdb746b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b 2023-02-09 14:56:11,868 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 76dd31bac2d74e73aa0cb36cfbdb746b 2023-02-09 14:56:11,868 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 14:56:11,869 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/692a8c7a466a4f28991fcee05ea8529e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e 2023-02-09 14:56:11,872 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 692a8c7a466a4f28991fcee05ea8529e 2023-02-09 14:56:11,872 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:56:11,873 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 932a93e1be1254f02a4c55d7755b4abb in 1346ms, sequenceid=23, compaction requested=true 2023-02-09 14:56:11,873 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:11,873 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:11,873 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 14:56:12,216 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/49802bec369c4accacaf0eafda9af8ff 2023-02-09 14:56:12,220 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49802bec369c4accacaf0eafda9af8ff 2023-02-09 14:56:12,220 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/17aa5eaa1ea644fc90f46b655ccedbba as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba 2023-02-09 14:56:12,243 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17aa5eaa1ea644fc90f46b655ccedbba 2023-02-09 14:56:12,244 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 14:56:12,244 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/49802bec369c4accacaf0eafda9af8ff as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff 2023-02-09 14:56:12,247 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49802bec369c4accacaf0eafda9af8ff 2023-02-09 14:56:12,248 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 14:56:12,248 INFO [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d22c6e5399984f7dc80f2f448d9fffcc in 1721ms, sequenceid=23, compaction requested=true 2023-02-09 14:56:12,248 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:12,248 DEBUG [rs(jenkins-hbase4.apache.org,43455,1675954416872)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:12,248 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 14:56:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 14:56:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 14:56:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 14:56:12,250 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,250 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:12,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:56:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:12,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43455,1675954416872' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 14:56:12,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d8674f6[Count = 0] remaining members to acquire global barrier 2023-02-09 14:56:12,252 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 14:56:12,252 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,254 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,254 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 14:56:12,255 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 14:56:12,255 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,43455,1675954416872' in zk 2023-02-09 14:56:12,255 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,255 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:56:12,256 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43455,1675954416872' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,43455,1675954416872': 2023-02-09 14:56:12,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43455,1675954416872' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 14:56:12,258 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 14:56:12,258 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:56:12,258 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:56:12,258 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 14:56:12,258 INFO [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:56:12,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:56:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:56:12,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:56:12,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:12,260 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:56:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,261 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:56:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 14:56:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:56:12,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:56:12,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:56:12,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:12,262 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 14:56:12,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 14:56:12,262 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:56:12,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:56:12,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:56:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 14:56:12,262 DEBUG [(jenkins-hbase4.apache.org,38213,1675954415718)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 14:56:12,263 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 14:56:12,363 DEBUG [Listener at localhost/33189] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:56:12,363 INFO [Listener at localhost/33189] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:12,363 INFO [Listener at localhost/33189] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:12,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:56:12,364 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:12,365 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): d22c6e5399984f7dc80f2f448d9fffcc/family1 is initiating major compaction (all files) 2023-02-09 14:56:12,365 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:12,365 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:12,367 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of d22c6e5399984f7dc80f2f448d9fffcc/family1 in testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:12,367 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp, totalSize=27.1 K 2023-02-09 14:56:12,367 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 08de665b63de4af1bddac1be722bdb07, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:12,368 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 4cf7089b868646a996e2ced78cbab76b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:12,368 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 9c9fbdc22de042e49d20a2f201eb2f4a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:12,368 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 1c3610fce6014c94991dce63294d747b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:12,369 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 17aa5eaa1ea644fc90f46b655ccedbba, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:12,377 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): d22c6e5399984f7dc80f2f448d9fffcc#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:12,816 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del 2023-02-09 14:56:12,816 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del 2023-02-09 14:56:13,670 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b8e781daecd4497b81ee35181b6f4a7c 2023-02-09 14:56:13,671 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family1/b8e781daecd4497b81ee35181b6f4a7c as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/b8e781daecd4497b81ee35181b6f4a7c 2023-02-09 14:56:13,695 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b8e781daecd4497b81ee35181b6f4a7c 2023-02-09 14:56:13,696 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d22c6e5399984f7dc80f2f448d9fffcc/family1 of d22c6e5399984f7dc80f2f448d9fffcc into b8e781daecd4497b81ee35181b6f4a7c(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:56:13,696 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:13,696 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:13,696 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): d22c6e5399984f7dc80f2f448d9fffcc/family2 is initiating major compaction (all files) 2023-02-09 14:56:13,697 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:13,697 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:13,697 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of d22c6e5399984f7dc80f2f448d9fffcc/family2 in testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:13,697 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp, totalSize=25.4 K 2023-02-09 14:56:13,698 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 7d6b7db223424a46b85f7c63f9658505, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:13,698 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 0085bbb1f6d340c5b738ea1aa0ce8f83, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:13,699 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 58382c35f68d4fa2afb96cdcad2d46bd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:13,699 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting cf52d117773241d784a30277d877a074, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:13,699 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 49802bec369c4accacaf0eafda9af8ff, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:13,709 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): d22c6e5399984f7dc80f2f448d9fffcc#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:13,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba] to archive 2023-02-09 14:56:13,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:13,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/08de665b63de4af1bddac1be722bdb07 2023-02-09 14:56:13,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/4cf7089b868646a996e2ced78cbab76b 2023-02-09 14:56:13,722 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209c70dc335332b4d7cbb8f4efd48a61084_del 2023-02-09 14:56:13,722 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209c70dc335332b4d7cbb8f4efd48a61084_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209c70dc335332b4d7cbb8f4efd48a61084_del 2023-02-09 14:56:13,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/9c9fbdc22de042e49d20a2f201eb2f4a 2023-02-09 14:56:13,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/1c3610fce6014c94991dce63294d747b 2023-02-09 14:56:13,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/17aa5eaa1ea644fc90f46b655ccedbba 2023-02-09 14:56:14,134 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ab99a9d6c9647239ef899b28dbb452b 2023-02-09 14:56:14,135 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/.tmp/family2/1ab99a9d6c9647239ef899b28dbb452b as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/1ab99a9d6c9647239ef899b28dbb452b 2023-02-09 14:56:14,138 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1ab99a9d6c9647239ef899b28dbb452b 2023-02-09 14:56:14,139 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d22c6e5399984f7dc80f2f448d9fffcc/family2 of d22c6e5399984f7dc80f2f448d9fffcc into 1ab99a9d6c9647239ef899b28dbb452b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:56:14,139 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:14,139 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:14,139 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 323cead325f2feb2545ff228bca36c06/family1 is initiating major compaction (all files) 2023-02-09 14:56:14,140 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:14,140 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:14,140 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 323cead325f2feb2545ff228bca36c06/family1 in testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:14,140 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp, totalSize=27.1 K 2023-02-09 14:56:14,141 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 0110ffd9694646c696adce49533864dc, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:14,141 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 33a7d0c9b74448e5a7766d6004843198, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:14,142 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting c12279a570424da1a035655fef9269f8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:14,142 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting cb9a55e742cb4f17b2daeb8dfbadd0a3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:14,142 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 3916ed0b8d074fc3b2d565f63181cd28, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:14,150 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 323cead325f2feb2545ff228bca36c06#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:14,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff] to archive 2023-02-09 14:56:14,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:14,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/7d6b7db223424a46b85f7c63f9658505 2023-02-09 14:56:14,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/0085bbb1f6d340c5b738ea1aa0ce8f83 2023-02-09 14:56:14,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/58382c35f68d4fa2afb96cdcad2d46bd 2023-02-09 14:56:14,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/cf52d117773241d784a30277d877a074 2023-02-09 14:56:14,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family2/49802bec369c4accacaf0eafda9af8ff 2023-02-09 14:56:14,588 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del 2023-02-09 14:56:14,588 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del 2023-02-09 14:56:15,460 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54f6ddf158f54ca1bf73590c758f32da 2023-02-09 14:56:15,461 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family1/54f6ddf158f54ca1bf73590c758f32da as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/54f6ddf158f54ca1bf73590c758f32da 2023-02-09 14:56:15,484 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54f6ddf158f54ca1bf73590c758f32da 2023-02-09 14:56:15,485 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 323cead325f2feb2545ff228bca36c06/family1 of 323cead325f2feb2545ff228bca36c06 into 54f6ddf158f54ca1bf73590c758f32da(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:56:15,485 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:15,486 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:15,486 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 323cead325f2feb2545ff228bca36c06/family2 is initiating major compaction (all files) 2023-02-09 14:56:15,486 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:15,486 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:15,486 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 323cead325f2feb2545ff228bca36c06/family2 in testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:15,486 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp, totalSize=25.4 K 2023-02-09 14:56:15,486 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 585f28e836ad4926af3a560b2d63c403, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:15,487 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting dcd0c61b06be492d82975a0865884a45, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:15,487 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting ff861395572d4cb1a85ace8b2e9aaeda, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:15,487 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 6f244e63e5fe45a8aa7ed2709c8a9277, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:15,488 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting a9e39d731dd145dd8b909e8823d473e1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:15,496 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 323cead325f2feb2545ff228bca36c06#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:15,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28] to archive 2023-02-09 14:56:15,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:15,519 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/0110ffd9694646c696adce49533864dc 2023-02-09 14:56:15,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/33a7d0c9b74448e5a7766d6004843198 2023-02-09 14:56:15,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/c12279a570424da1a035655fef9269f8 2023-02-09 14:56:15,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/cb9a55e742cb4f17b2daeb8dfbadd0a3 2023-02-09 14:56:15,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/3916ed0b8d074fc3b2d565f63181cd28 2023-02-09 14:56:15,912 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302099e3ab5d61fce4ab0bb1c16acac90f37f_del 2023-02-09 14:56:15,913 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302099e3ab5d61fce4ab0bb1c16acac90f37f_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302099e3ab5d61fce4ab0bb1c16acac90f37f_del 2023-02-09 14:56:16,726 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5ab4128bc334dec87fc4979e9b69fd7 2023-02-09 14:56:16,727 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/.tmp/family2/d5ab4128bc334dec87fc4979e9b69fd7 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/d5ab4128bc334dec87fc4979e9b69fd7 2023-02-09 14:56:16,730 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d5ab4128bc334dec87fc4979e9b69fd7 2023-02-09 14:56:16,731 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 323cead325f2feb2545ff228bca36c06/family2 of 323cead325f2feb2545ff228bca36c06 into d5ab4128bc334dec87fc4979e9b69fd7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:56:16,731 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:16,731 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:16,731 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 932a93e1be1254f02a4c55d7755b4abb/family1 is initiating major compaction (all files) 2023-02-09 14:56:16,731 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:16,731 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:16,731 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 932a93e1be1254f02a4c55d7755b4abb/family1 in testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:16,732 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp, totalSize=27.1 K 2023-02-09 14:56:16,732 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting f67ec26153b24ae4aa08b859e206841c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:16,732 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 45a1150e467f430cab93d6fc89ab6f0f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:16,733 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting bbfa873286414797a1d2f81d76e9bd55, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:16,733 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting e391328208da4cc09de5aecd185ea231, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:16,733 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 76dd31bac2d74e73aa0cb36cfbdb746b, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:16,744 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 932a93e1be1254f02a4c55d7755b4abb#family1#compaction#201 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:16,783 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del 2023-02-09 14:56:16,783 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del 2023-02-09 14:56:16,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1] to archive 2023-02-09 14:56:16,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:16,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/585f28e836ad4926af3a560b2d63c403 2023-02-09 14:56:16,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/dcd0c61b06be492d82975a0865884a45 2023-02-09 14:56:16,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/ff861395572d4cb1a85ace8b2e9aaeda 2023-02-09 14:56:16,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/6f244e63e5fe45a8aa7ed2709c8a9277 2023-02-09 14:56:16,825 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family2/a9e39d731dd145dd8b909e8823d473e1 2023-02-09 14:56:17,248 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f246365086748daa89362b5b1137eea 2023-02-09 14:56:17,249 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family1/2f246365086748daa89362b5b1137eea as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/2f246365086748daa89362b5b1137eea 2023-02-09 14:56:17,272 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f246365086748daa89362b5b1137eea 2023-02-09 14:56:17,273 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 932a93e1be1254f02a4c55d7755b4abb/family1 of 932a93e1be1254f02a4c55d7755b4abb into 2f246365086748daa89362b5b1137eea(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:56:17,273 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:17,273 DEBUG [Listener at localhost/33189] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 14:56:17,273 DEBUG [Listener at localhost/33189] regionserver.HStore(1912): 932a93e1be1254f02a4c55d7755b4abb/family2 is initiating major compaction (all files) 2023-02-09 14:56:17,273 INFO [Listener at localhost/33189] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:56:17,274 INFO [Listener at localhost/33189] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:56:17,274 INFO [Listener at localhost/33189] regionserver.HRegion(2258): Starting compaction of 932a93e1be1254f02a4c55d7755b4abb/family2 in testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:17,274 INFO [Listener at localhost/33189] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e] into tmpdir=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp, totalSize=25.4 K 2023-02-09 14:56:17,275 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 57ada83452454b8ab55dc1ef7faa160a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675954562854 2023-02-09 14:56:17,275 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 0ecda03151b543c9bd64dd2fdff8ce58, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675954564739 2023-02-09 14:56:17,276 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting e2bd84fc264c410eb41326c3522047cd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675954566631 2023-02-09 14:56:17,276 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting ea7bd90e35d34544821f93c18de7ff72, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675954568519 2023-02-09 14:56:17,276 DEBUG [Listener at localhost/33189] compactions.Compactor(207): Compacting 692a8c7a466a4f28991fcee05ea8529e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 14:56:17,285 INFO [Listener at localhost/33189] throttle.PressureAwareThroughputController(145): 932a93e1be1254f02a4c55d7755b4abb#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:56:17,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b] to archive 2023-02-09 14:56:17,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:17,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/f67ec26153b24ae4aa08b859e206841c 2023-02-09 14:56:17,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/45a1150e467f430cab93d6fc89ab6f0f 2023-02-09 14:56:17,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/bbfa873286414797a1d2f81d76e9bd55 2023-02-09 14:56:17,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/e391328208da4cc09de5aecd185ea231 2023-02-09 14:56:17,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/76dd31bac2d74e73aa0cb36cfbdb746b 2023-02-09 14:56:17,695 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209532f22209e2f4a038bae5f732afa2024_del 2023-02-09 14:56:17,696 INFO [Listener at localhost/33189] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209532f22209e2f4a038bae5f732afa2024_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209532f22209e2f4a038bae5f732afa2024_del 2023-02-09 14:56:18,509 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a800e8cbbbc42aa8a1584ee12f95833 2023-02-09 14:56:18,510 DEBUG [Listener at localhost/33189] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/.tmp/family2/7a800e8cbbbc42aa8a1584ee12f95833 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/7a800e8cbbbc42aa8a1584ee12f95833 2023-02-09 14:56:18,513 INFO [Listener at localhost/33189] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a800e8cbbbc42aa8a1584ee12f95833 2023-02-09 14:56:18,514 INFO [Listener at localhost/33189] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 932a93e1be1254f02a4c55d7755b4abb/family2 of 932a93e1be1254f02a4c55d7755b4abb into 7a800e8cbbbc42aa8a1584ee12f95833(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 14:56:18,514 DEBUG [Listener at localhost/33189] regionserver.HRegion(2288): Compaction status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:18,517 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:18,520 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:18,523 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:18,524 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:18,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e] to archive 2023-02-09 14:56:18,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:18,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/57ada83452454b8ab55dc1ef7faa160a 2023-02-09 14:56:18,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/0ecda03151b543c9bd64dd2fdff8ce58 2023-02-09 14:56:18,531 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x19UB\xE3N\xDEs\xBA\xC6\xFC\x10\x1C '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-09 14:56:18,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/e2bd84fc264c410eb41326c3522047cd 2023-02-09 14:56:18,535 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/ea7bd90e35d34544821f93c18de7ff72 2023-02-09 14:56:18,537 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family2/692a8c7a466a4f28991fcee05ea8529e 2023-02-09 14:56:18,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 14:56:18,555 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 14:56:18,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:56:18,555 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675955178555 2023-02-09 14:56:18,556 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 14:56:18,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-09 14:56:18,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del 2023-02-09 14:56:18,961 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del 2023-02-09 14:56:18,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del 2023-02-09 14:56:18,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:18,985 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] zookeeper.ReadOnlyZKClient(139): Connect 0x182d8141 to 127.0.0.1:56347 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:56:18,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c7730d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:56:18,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:19,148 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209c9e9183b02524779bdc70bd9c265c9df to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209c9e9183b02524779bdc70bd9c265c9df 2023-02-09 14:56:19,149 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:19,160 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:56:19,163 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53794, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:56:19,168 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:19,168 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:19,169 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:56:19,171 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55154, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:56:19,202 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/639a476278a647258aff688b98ac63ee first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 14:56:19,203 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/639a476278a647258aff688b98ac63ee}] 2023-02-09 14:56:19,209 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/639a476278a647258aff688b98ac63ee for inclusion in d22c6e5399984f7dc80f2f448d9fffcc/family1 2023-02-09 14:56:19,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 14:56:19,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 14:56:19,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for d22c6e5399984f7dc80f2f448d9fffcc: 2023-02-09 14:56:19,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/639a476278a647258aff688b98ac63ee to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__lgf04f6bkut80emt21a35l4msji0dj65vakjjn8fe6cb2jo2715vvvhqb93jbmma/family1/639a476278a647258aff688b98ac63ee 2023-02-09 14:56:19,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__lgf04f6bkut80emt21a35l4msji0dj65vakjjn8fe6cb2jo2715vvvhqb93jbmma/family1/639a476278a647258aff688b98ac63ee as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/375926de20df4a098a5f4efb4d4bae36_SeqId_28_ 2023-02-09 14:56:19,239 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__lgf04f6bkut80emt21a35l4msji0dj65vakjjn8fe6cb2jo2715vvvhqb93jbmma/family1/639a476278a647258aff688b98ac63ee into d22c6e5399984f7dc80f2f448d9fffcc/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/375926de20df4a098a5f4efb4d4bae36_SeqId_28_ - updating store file list. 2023-02-09 14:56:19,266 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/375926de20df4a098a5f4efb4d4bae36_SeqId_28_ into d22c6e5399984f7dc80f2f448d9fffcc/family1 2023-02-09 14:56:19,267 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__lgf04f6bkut80emt21a35l4msji0dj65vakjjn8fe6cb2jo2715vvvhqb93jbmma/family1/639a476278a647258aff688b98ac63ee into d22c6e5399984f7dc80f2f448d9fffcc/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/family1/375926de20df4a098a5f4efb4d4bae36_SeqId_28_) 2023-02-09 14:56:19,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__lgf04f6bkut80emt21a35l4msji0dj65vakjjn8fe6cb2jo2715vvvhqb93jbmma/family1/639a476278a647258aff688b98ac63ee 2023-02-09 14:56:19,269 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:19,270 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:19,271 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209272fbc0391f84973bd6b4075e09ba731 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209272fbc0391f84973bd6b4075e09ba731 2023-02-09 14:56:19,272 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209340d30c6a51b450999733056cd8aed68 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209340d30c6a51b450999733056cd8aed68 2023-02-09 14:56:19,273 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095258d8e775384c90921910c37fbaa28e to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095258d8e775384c90921910c37fbaa28e 2023-02-09 14:56:19,274 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209ff3305da875d41aab125be7a31171d2d to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209ff3305da875d41aab125be7a31171d2d 2023-02-09 14:56:19,274 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 14:56:19,274 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 14:56:19,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:19,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:19,378 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:56:19,452 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209d0dd890ac0d24ed5a360b6def61c67d1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209d0dd890ac0d24ed5a360b6def61c67d1 2023-02-09 14:56:19,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:19,468 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:19,468 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:19,509 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/b0aa5c5fbd984993b161100b2fbc8a61 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 14:56:19,510 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/b0aa5c5fbd984993b161100b2fbc8a61}] 2023-02-09 14:56:19,512 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/b0aa5c5fbd984993b161100b2fbc8a61 for inclusion in 323cead325f2feb2545ff228bca36c06/family1 2023-02-09 14:56:19,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 14:56:19,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 14:56:19,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 323cead325f2feb2545ff228bca36c06: 2023-02-09 14:56:19,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/b0aa5c5fbd984993b161100b2fbc8a61 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__c5hq7bvhofb3q6nlln1gkdvul1iq0nm6jn9bdt1g9nuou4rr4o3siso37r9b45qp/family1/b0aa5c5fbd984993b161100b2fbc8a61 2023-02-09 14:56:19,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__c5hq7bvhofb3q6nlln1gkdvul1iq0nm6jn9bdt1g9nuou4rr4o3siso37r9b45qp/family1/b0aa5c5fbd984993b161100b2fbc8a61 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/770b03e879d64354a6b5f82595954066_SeqId_28_ 2023-02-09 14:56:19,538 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__c5hq7bvhofb3q6nlln1gkdvul1iq0nm6jn9bdt1g9nuou4rr4o3siso37r9b45qp/family1/b0aa5c5fbd984993b161100b2fbc8a61 into 323cead325f2feb2545ff228bca36c06/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/770b03e879d64354a6b5f82595954066_SeqId_28_ - updating store file list. 2023-02-09 14:56:19,562 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/770b03e879d64354a6b5f82595954066_SeqId_28_ into 323cead325f2feb2545ff228bca36c06/family1 2023-02-09 14:56:19,563 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__c5hq7bvhofb3q6nlln1gkdvul1iq0nm6jn9bdt1g9nuou4rr4o3siso37r9b45qp/family1/b0aa5c5fbd984993b161100b2fbc8a61 into 323cead325f2feb2545ff228bca36c06/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/family1/770b03e879d64354a6b5f82595954066_SeqId_28_) 2023-02-09 14:56:19,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__c5hq7bvhofb3q6nlln1gkdvul1iq0nm6jn9bdt1g9nuou4rr4o3siso37r9b45qp/family1/b0aa5c5fbd984993b161100b2fbc8a61 2023-02-09 14:56:19,565 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:19,566 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:19,567 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209644da45b9399457f8245612b9b60b6fa to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209644da45b9399457f8245612b9b60b6fa 2023-02-09 14:56:19,568 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302096bf6829f1ea944a9a6f2c30715aecacb to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302096bf6829f1ea944a9a6f2c30715aecacb 2023-02-09 14:56:19,569 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209eb760f79302040dba2ca05773ae19c3f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209eb760f79302040dba2ca05773ae19c3f 2023-02-09 14:56:19,571 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ecd01cd61ec443f19500d621ae3e8e21 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ecd01cd61ec443f19500d621ae3e8e21 2023-02-09 14:56:19,571 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 14:56:19,571 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 14:56:19,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:19,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:19,684 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:56:19,684 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5e663a06 to 127.0.0.1:56347 2023-02-09 14:56:19,685 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:20,560 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209e6249fe3d3a34f93b5fd78383660a100 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209e6249fe3d3a34f93b5fd78383660a100 2023-02-09 14:56:20,561 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:20,576 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:56:20,576 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:56:20,605 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/6cbdca916ea842098fb1e183be8e98e1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 14:56:20,606 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb., hostname=jenkins-hbase4.apache.org,43455,1675954416872, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/6cbdca916ea842098fb1e183be8e98e1}] 2023-02-09 14:56:20,608 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(763): Validating hfile at hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/6cbdca916ea842098fb1e183be8e98e1 for inclusion in 932a93e1be1254f02a4c55d7755b4abb/family1 2023-02-09 14:56:20,624 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:56:20,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 14:56:20,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 14:56:20,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegion(2445): Flush status journal for 932a93e1be1254f02a4c55d7755b4abb: 2023-02-09 14:56:20,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/6cbdca916ea842098fb1e183be8e98e1 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__bd91e3mm8k46huf5arjnurklfet7mtua864epgmmikfgnqoli0rorgbo0eq16epf/family1/6cbdca916ea842098fb1e183be8e98e1 2023-02-09 14:56:20,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__bd91e3mm8k46huf5arjnurklfet7mtua864epgmmikfgnqoli0rorgbo0eq16epf/family1/6cbdca916ea842098fb1e183be8e98e1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/2f972a1b750c432c8f75e1cfd8f49423_SeqId_28_ 2023-02-09 14:56:20,633 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(849): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__bd91e3mm8k46huf5arjnurklfet7mtua864epgmmikfgnqoli0rorgbo0eq16epf/family1/6cbdca916ea842098fb1e183be8e98e1 into 932a93e1be1254f02a4c55d7755b4abb/family1 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/2f972a1b750c432c8f75e1cfd8f49423_SeqId_28_ - updating store file list. 2023-02-09 14:56:20,655 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(882): Loaded HFile hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/2f972a1b750c432c8f75e1cfd8f49423_SeqId_28_ into 932a93e1be1254f02a4c55d7755b4abb/family1 2023-02-09 14:56:20,655 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.HStore(855): Successfully loaded hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__bd91e3mm8k46huf5arjnurklfet7mtua864epgmmikfgnqoli0rorgbo0eq16epf/family1/6cbdca916ea842098fb1e183be8e98e1 into 932a93e1be1254f02a4c55d7755b4abb/family1 (new location: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/family1/2f972a1b750c432c8f75e1cfd8f49423_SeqId_28_) 2023-02-09 14:56:20,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43455] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/staging/jenkins__testMajorCompactionFromAdmin__bd91e3mm8k46huf5arjnurklfet7mtua864epgmmikfgnqoli0rorgbo0eq16epf/family1/6cbdca916ea842098fb1e183be8e98e1 2023-02-09 14:56:20,657 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:20,658 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:20,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302091b6832b23d11447e9e1cc82335a8b7fe to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302091b6832b23d11447e9e1cc82335a8b7fe 2023-02-09 14:56:20,660 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302093a72bbc8d156440ab221a4d8cb7630c0 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302093a72bbc8d156440ab221a4d8cb7630c0 2023-02-09 14:56:20,661 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ee7f3a55e10d4c37a2cbfd24737e356f to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ee7f3a55e10d4c37a2cbfd24737e356f 2023-02-09 14:56:20,661 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209fd5b70b485424570b91baf01c6231c74 to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209fd5b70b485424570b91baf01c6231c74 2023-02-09 14:56:20,661 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 14:56:20,661 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 14:56:20,662 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:20,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 14:56:20,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 14:56:20,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del] 2023-02-09 14:56:20,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:20,663 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209f66ddd10ef9d46249fdcbaddb9268275_del 2023-02-09 14:56:20,663 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 14:56:20,663 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del] 2023-02-09 14:56:20,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:20,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ed99491d619243eeb543f2f0fee6bf1c_del 2023-02-09 14:56:20,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 14:56:20,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] compactions.PartitionedMobCompactor(367): [hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del] 2023-02-09 14:56:20,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:56:20,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del to hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302090b22d3f5fe5949ec85dd679f4b5e9d80_del 2023-02-09 14:56:20,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 14:56:20,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675954420369] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 14:56:20,668 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 14:56:20,669 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.1150 sec 2023-02-09 14:56:20,681 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:20,683 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:20,685 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 14:56:20,686 DEBUG [Listener at localhost/33189] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 14:56:20,690 INFO [Listener at localhost/33189] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:56:20,725 INFO [Listener at localhost/33189] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=251 (was 259), OpenFileDescriptor=640 (was 664), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 81) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3169 (was 3031) - AvailableMemoryMB LEAK? - 2023-02-09 14:56:20,726 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x480b078d to 127.0.0.1:56347 2023-02-09 14:56:20,726 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:20,726 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 14:56:20,726 INFO [Listener at localhost/33189] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:56:20,726 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57919e99 to 127.0.0.1:56347 2023-02-09 14:56:20,726 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:20,727 DEBUG [Listener at localhost/33189] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 14:56:20,727 DEBUG [Listener at localhost/33189] util.JVMClusterUtil(257): Found active master hash=780862675, stopped=false 2023-02-09 14:56:20,727 INFO [Listener at localhost/33189] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:56:20,729 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:56:20,729 INFO [Listener at localhost/33189] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:56:20,729 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:56:20,729 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:56:20,729 DEBUG [Listener at localhost/33189] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x270393bd to 127.0.0.1:56347 2023-02-09 14:56:20,729 DEBUG [Listener at localhost/33189] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:20,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:56:20,730 INFO [Listener at localhost/33189] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43455,1675954416872' ***** 2023-02-09 14:56:20,730 INFO [Listener at localhost/33189] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 14:56:20,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:56:20,730 INFO [RS:0;jenkins-hbase4:43455] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 14:56:20,730 INFO [RS:0;jenkins-hbase4:43455] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 14:56:20,730 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 14:56:20,730 INFO [RS:0;jenkins-hbase4:43455] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 14:56:20,731 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for bea37efa3b23a35293273de94eb1d3d2 2023-02-09 14:56:20,731 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 55b8d1f0f9f6d2db14ef80187e2488f8 2023-02-09 14:56:20,731 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:56:20,731 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for c077c04bfcb4d7160a27e1f318e339f6 2023-02-09 14:56:20,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bea37efa3b23a35293273de94eb1d3d2, disabling compactions & flushes 2023-02-09 14:56:20,731 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 97b6b9df89d0f2a426e3ed381519a6e4 2023-02-09 14:56:20,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for a92002b17d4c5e771b3092e4c4816fd2 2023-02-09 14:56:20,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 9a00c85257e134deaed37338dc2b5000 2023-02-09 14:56:20,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. after waiting 0 ms 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 003f0df30fea469fcf63c1030019683b 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 0f4e5cd8c7b1d17e86c4691d686f3baf 2023-02-09 14:56:20,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 9a6c61f775706f7023a02aced32af95c 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for d22c6e5399984f7dc80f2f448d9fffcc 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for bb3d06ff69b5e78251fb182a52cc2b7b 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 4a620fe72a6269f78f560319e3947df4 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 45873f0ac0c88ebf31d0fcc61f3d2515 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 773391e6c583b5ff2a5d0746351f3fed 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 431c1f7d6260ecba176dba7e512abdb1 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 323cead325f2feb2545ff228bca36c06 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for b989787f43967a90c0c5150dc576648d 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 482df8fbf041dd5fc164a0c315c9e3ed 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 932a93e1be1254f02a4c55d7755b4abb 2023-02-09 14:56:20,732 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:20,732 DEBUG [RS:0;jenkins-hbase4:43455] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13a2099f to 127.0.0.1:56347 2023-02-09 14:56:20,732 DEBUG [RS:0;jenkins-hbase4:43455] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:20,733 INFO [RS:0;jenkins-hbase4:43455] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 14:56:20,733 INFO [RS:0;jenkins-hbase4:43455] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 14:56:20,733 INFO [RS:0;jenkins-hbase4:43455] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 14:56:20,733 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 14:56:20,733 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-09 14:56:20,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 14:56:20,733 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 14:56:20,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 14:56:20,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 14:56:20,733 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 14:56:20,733 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-09 14:56:20,734 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 003f0df30fea469fcf63c1030019683b, 0f4e5cd8c7b1d17e86c4691d686f3baf, 1588230740, 323cead325f2feb2545ff228bca36c06, 431c1f7d6260ecba176dba7e512abdb1, 45873f0ac0c88ebf31d0fcc61f3d2515, 482df8fbf041dd5fc164a0c315c9e3ed, 4a620fe72a6269f78f560319e3947df4, 55b8d1f0f9f6d2db14ef80187e2488f8, 773391e6c583b5ff2a5d0746351f3fed, 932a93e1be1254f02a4c55d7755b4abb, 97b6b9df89d0f2a426e3ed381519a6e4, 9a00c85257e134deaed37338dc2b5000, 9a6c61f775706f7023a02aced32af95c, a92002b17d4c5e771b3092e4c4816fd2, b989787f43967a90c0c5150dc576648d, bb3d06ff69b5e78251fb182a52cc2b7b, bea37efa3b23a35293273de94eb1d3d2, c077c04bfcb4d7160a27e1f318e339f6, d22c6e5399984f7dc80f2f448d9fffcc, e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:56:20,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithMonthlyPolicy/bea37efa3b23a35293273de94eb1d3d2/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-09 14:56:20,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bea37efa3b23a35293273de94eb1d3d2: Waiting for close lock at 1675954580731Running coprocessor pre-close hooks at 1675954580731Disabling compacts and flushes for region at 1675954580731Disabling writes for close at 1675954580732 (+1 ms)Writing region close event to WAL at 1675954580733 (+1 ms)Running coprocessor post-close hooks at 1675954580736 (+3 ms)Closed at 1675954580737 (+1 ms) 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1675954433348.bea37efa3b23a35293273de94eb1d3d2. 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 55b8d1f0f9f6d2db14ef80187e2488f8, disabling compactions & flushes 2023-02-09 14:56:20,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. after waiting 0 ms 2023-02-09 14:56:20,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:56:20,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/55b8d1f0f9f6d2db14ef80187e2488f8/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 14:56:20,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 55b8d1f0f9f6d2db14ef80187e2488f8: Waiting for close lock at 1675954580737Running coprocessor pre-close hooks at 1675954580737Disabling compacts and flushes for region at 1675954580737Disabling writes for close at 1675954580737Writing region close event to WAL at 1675954580737Running coprocessor post-close hooks at 1675954580740 (+3 ms)Closed at 1675954580741 (+1 ms) 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1675954453536.55b8d1f0f9f6d2db14ef80187e2488f8. 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e5181eaff4f036103b5f31d3d6a127e7, disabling compactions & flushes 2023-02-09 14:56:20,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. after waiting 0 ms 2023-02-09 14:56:20,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:56:20,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e5181eaff4f036103b5f31d3d6a127e7 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-09 14:56:20,774 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:56:20,816 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-09 14:56:20,934 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 003f0df30fea469fcf63c1030019683b, 0f4e5cd8c7b1d17e86c4691d686f3baf, 1588230740, 323cead325f2feb2545ff228bca36c06, 431c1f7d6260ecba176dba7e512abdb1, 45873f0ac0c88ebf31d0fcc61f3d2515, 482df8fbf041dd5fc164a0c315c9e3ed, 4a620fe72a6269f78f560319e3947df4, 773391e6c583b5ff2a5d0746351f3fed, 932a93e1be1254f02a4c55d7755b4abb, 97b6b9df89d0f2a426e3ed381519a6e4, 9a00c85257e134deaed37338dc2b5000, 9a6c61f775706f7023a02aced32af95c, a92002b17d4c5e771b3092e4c4816fd2, b989787f43967a90c0c5150dc576648d, bb3d06ff69b5e78251fb182a52cc2b7b, c077c04bfcb4d7160a27e1f318e339f6, d22c6e5399984f7dc80f2f448d9fffcc, e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:56:21,059 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 14:56:21,059 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 14:56:21,134 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 003f0df30fea469fcf63c1030019683b, 0f4e5cd8c7b1d17e86c4691d686f3baf, 1588230740, 323cead325f2feb2545ff228bca36c06, 431c1f7d6260ecba176dba7e512abdb1, 45873f0ac0c88ebf31d0fcc61f3d2515, 482df8fbf041dd5fc164a0c315c9e3ed, 4a620fe72a6269f78f560319e3947df4, 773391e6c583b5ff2a5d0746351f3fed, 932a93e1be1254f02a4c55d7755b4abb, 97b6b9df89d0f2a426e3ed381519a6e4, 9a00c85257e134deaed37338dc2b5000, 9a6c61f775706f7023a02aced32af95c, a92002b17d4c5e771b3092e4c4816fd2, b989787f43967a90c0c5150dc576648d, bb3d06ff69b5e78251fb182a52cc2b7b, c077c04bfcb4d7160a27e1f318e339f6, d22c6e5399984f7dc80f2f448d9fffcc, e5181eaff4f036103b5f31d3d6a127e7 2023-02-09 14:56:21,146 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/info/4895082d295942089198f9a4c2f3ceb9 2023-02-09 14:56:21,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4895082d295942089198f9a4c2f3ceb9 2023-02-09 14:56:21,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/.tmp/info/4f83e699000f4b09aba5fd92651a2074 2023-02-09 14:56:21,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/.tmp/info/4f83e699000f4b09aba5fd92651a2074 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/info/4f83e699000f4b09aba5fd92651a2074 2023-02-09 14:56:21,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/info/4f83e699000f4b09aba5fd92651a2074, entries=3, sequenceid=7, filesize=4.8 K 2023-02-09 14:56:21,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for e5181eaff4f036103b5f31d3d6a127e7 in 421ms, sequenceid=7, compaction requested=false 2023-02-09 14:56:21,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/namespace/e5181eaff4f036103b5f31d3d6a127e7/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-09 14:56:21,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e5181eaff4f036103b5f31d3d6a127e7: Waiting for close lock at 1675954580741Running coprocessor pre-close hooks at 1675954580741Disabling compacts and flushes for region at 1675954580741Disabling writes for close at 1675954580741Obtaining lock to block concurrent updates at 1675954580741Preparing flush snapshotting stores in e5181eaff4f036103b5f31d3d6a127e7 at 1675954580741Finished memstore snapshotting hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1675954580741Flushing stores of hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. at 1675954580742 (+1 ms)Flushing e5181eaff4f036103b5f31d3d6a127e7/info: creating writer at 1675954580742Flushing e5181eaff4f036103b5f31d3d6a127e7/info: appending metadata at 1675954580752 (+10 ms)Flushing e5181eaff4f036103b5f31d3d6a127e7/info: closing flushed file at 1675954580752Flushing e5181eaff4f036103b5f31d3d6a127e7/info: reopening flushed file at 1675954581159 (+407 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for e5181eaff4f036103b5f31d3d6a127e7 in 421ms, sequenceid=7, compaction requested=false at 1675954581162 (+3 ms)Writing region close event to WAL at 1675954581164 (+2 ms)Running coprocessor post-close hooks at 1675954581166 (+2 ms)Closed at 1675954581167 (+1 ms) 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675954418954.e5181eaff4f036103b5f31d3d6a127e7. 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c077c04bfcb4d7160a27e1f318e339f6, disabling compactions & flushes 2023-02-09 14:56:21,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. after waiting 0 ms 2023-02-09 14:56:21,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:56:21,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/c077c04bfcb4d7160a27e1f318e339f6/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 14:56:21,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:56:21,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c077c04bfcb4d7160a27e1f318e339f6: Waiting for close lock at 1675954581167Running coprocessor pre-close hooks at 1675954581167Disabling compacts and flushes for region at 1675954581167Disabling writes for close at 1675954581167Writing region close event to WAL at 1675954581167Running coprocessor post-close hooks at 1675954581170 (+3 ms)Closed at 1675954581170 2023-02-09 14:56:21,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1675954453536.c077c04bfcb4d7160a27e1f318e339f6. 2023-02-09 14:56:21,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 97b6b9df89d0f2a426e3ed381519a6e4, disabling compactions & flushes 2023-02-09 14:56:21,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:56:21,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:56:21,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. after waiting 0 ms 2023-02-09 14:56:21,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:56:21,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/97b6b9df89d0f2a426e3ed381519a6e4/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 14:56:21,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 97b6b9df89d0f2a426e3ed381519a6e4: Waiting for close lock at 1675954581170Running coprocessor pre-close hooks at 1675954581170Disabling compacts and flushes for region at 1675954581170Disabling writes for close at 1675954581171 (+1 ms)Writing region close event to WAL at 1675954581171Running coprocessor post-close hooks at 1675954581173 (+2 ms)Closed at 1675954581174 (+1 ms) 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675954483810.97b6b9df89d0f2a426e3ed381519a6e4. 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a92002b17d4c5e771b3092e4c4816fd2, disabling compactions & flushes 2023-02-09 14:56:21,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. after waiting 0 ms 2023-02-09 14:56:21,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:56:21,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/a92002b17d4c5e771b3092e4c4816fd2/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a92002b17d4c5e771b3092e4c4816fd2: Waiting for close lock at 1675954581174Running coprocessor pre-close hooks at 1675954581174Disabling compacts and flushes for region at 1675954581174Disabling writes for close at 1675954581174Writing region close event to WAL at 1675954581175 (+1 ms)Running coprocessor post-close hooks at 1675954581177 (+2 ms)Closed at 1675954581177 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1675954532528.a92002b17d4c5e771b3092e4c4816fd2. 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a00c85257e134deaed37338dc2b5000, disabling compactions & flushes 2023-02-09 14:56:21,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. after waiting 0 ms 2023-02-09 14:56:21,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:56:21,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/9a00c85257e134deaed37338dc2b5000/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a00c85257e134deaed37338dc2b5000: Waiting for close lock at 1675954581177Running coprocessor pre-close hooks at 1675954581177Disabling compacts and flushes for region at 1675954581177Disabling writes for close at 1675954581177Writing region close event to WAL at 1675954581178 (+1 ms)Running coprocessor post-close hooks at 1675954581180 (+2 ms)Closed at 1675954581181 (+1 ms) 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1675954532528.9a00c85257e134deaed37338dc2b5000. 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 003f0df30fea469fcf63c1030019683b, disabling compactions & flushes 2023-02-09 14:56:21,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. after waiting 0 ms 2023-02-09 14:56:21,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:56:21,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/ns/testMinorCompaction/003f0df30fea469fcf63c1030019683b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:56:21,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 003f0df30fea469fcf63c1030019683b: Waiting for close lock at 1675954581181Running coprocessor pre-close hooks at 1675954581181Disabling compacts and flushes for region at 1675954581181Disabling writes for close at 1675954581181Writing region close event to WAL at 1675954581182 (+1 ms)Running coprocessor post-close hooks at 1675954581184 (+2 ms)Closed at 1675954581184 2023-02-09 14:56:21,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1675954532528.003f0df30fea469fcf63c1030019683b. 2023-02-09 14:56:21,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0f4e5cd8c7b1d17e86c4691d686f3baf, disabling compactions & flushes 2023-02-09 14:56:21,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:56:21,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:56:21,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. after waiting 0 ms 2023-02-09 14:56:21,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:56:21,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/0f4e5cd8c7b1d17e86c4691d686f3baf/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 14:56:21,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:56:21,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0f4e5cd8c7b1d17e86c4691d686f3baf: Waiting for close lock at 1675954581184Running coprocessor pre-close hooks at 1675954581184Disabling compacts and flushes for region at 1675954581185 (+1 ms)Disabling writes for close at 1675954581185Writing region close event to WAL at 1675954581186 (+1 ms)Running coprocessor post-close hooks at 1675954581188 (+2 ms)Closed at 1675954581188 2023-02-09 14:56:21,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1675954506571.0f4e5cd8c7b1d17e86c4691d686f3baf. 2023-02-09 14:56:21,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a6c61f775706f7023a02aced32af95c, disabling compactions & flushes 2023-02-09 14:56:21,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:56:21,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:56:21,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. after waiting 0 ms 2023-02-09 14:56:21,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:56:21,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithWeeklyPolicy/9a6c61f775706f7023a02aced32af95c/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-09 14:56:21,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:56:21,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a6c61f775706f7023a02aced32af95c: Waiting for close lock at 1675954581189Running coprocessor pre-close hooks at 1675954581189Disabling compacts and flushes for region at 1675954581189Disabling writes for close at 1675954581189Writing region close event to WAL at 1675954581190 (+1 ms)Running coprocessor post-close hooks at 1675954581193 (+3 ms)Closed at 1675954581193 2023-02-09 14:56:21,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1675954444739.9a6c61f775706f7023a02aced32af95c. 2023-02-09 14:56:21,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d22c6e5399984f7dc80f2f448d9fffcc, disabling compactions & flushes 2023-02-09 14:56:21,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:21,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:21,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. after waiting 0 ms 2023-02-09 14:56:21,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:21,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/d22c6e5399984f7dc80f2f448d9fffcc/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:21,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d22c6e5399984f7dc80f2f448d9fffcc: Waiting for close lock at 1675954581194Running coprocessor pre-close hooks at 1675954581194Disabling compacts and flushes for region at 1675954581194Disabling writes for close at 1675954581194Writing region close event to WAL at 1675954581194Running coprocessor post-close hooks at 1675954581197 (+3 ms)Closed at 1675954581197 2023-02-09 14:56:21,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1675954560707.d22c6e5399984f7dc80f2f448d9fffcc. 2023-02-09 14:56:21,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bb3d06ff69b5e78251fb182a52cc2b7b, disabling compactions & flushes 2023-02-09 14:56:21,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:56:21,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:56:21,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. after waiting 0 ms 2023-02-09 14:56:21,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:56:21,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/bb3d06ff69b5e78251fb182a52cc2b7b/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 14:56:21,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bb3d06ff69b5e78251fb182a52cc2b7b: Waiting for close lock at 1675954581197Running coprocessor pre-close hooks at 1675954581197Disabling compacts and flushes for region at 1675954581197Disabling writes for close at 1675954581198 (+1 ms)Writing region close event to WAL at 1675954581198Running coprocessor post-close hooks at 1675954581201 (+3 ms)Closed at 1675954581201 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1675954506571.bb3d06ff69b5e78251fb182a52cc2b7b. 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4a620fe72a6269f78f560319e3947df4, disabling compactions & flushes 2023-02-09 14:56:21,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. after waiting 0 ms 2023-02-09 14:56:21,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:56:21,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMinorCompactionWithMonthlyPolicy/4a620fe72a6269f78f560319e3947df4/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-09 14:56:21,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4a620fe72a6269f78f560319e3947df4: Waiting for close lock at 1675954581201Running coprocessor pre-close hooks at 1675954581201Disabling compacts and flushes for region at 1675954581201Disabling writes for close at 1675954581201Writing region close event to WAL at 1675954581202 (+1 ms)Running coprocessor post-close hooks at 1675954581204 (+2 ms)Closed at 1675954581205 (+1 ms) 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1675954550248.4a620fe72a6269f78f560319e3947df4. 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 45873f0ac0c88ebf31d0fcc61f3d2515, disabling compactions & flushes 2023-02-09 14:56:21,205 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. after waiting 0 ms 2023-02-09 14:56:21,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testCompactionWithHFileLink/45873f0ac0c88ebf31d0fcc61f3d2515/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 14:56:21,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 45873f0ac0c88ebf31d0fcc61f3d2515: Waiting for close lock at 1675954581205Running coprocessor pre-close hooks at 1675954581205Disabling compacts and flushes for region at 1675954581205Disabling writes for close at 1675954581205Writing region close event to WAL at 1675954581206 (+1 ms)Running coprocessor post-close hooks at 1675954581208 (+2 ms)Closed at 1675954581208 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1675954506571.45873f0ac0c88ebf31d0fcc61f3d2515. 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 773391e6c583b5ff2a5d0746351f3fed, disabling compactions & flushes 2023-02-09 14:56:21,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. after waiting 0 ms 2023-02-09 14:56:21,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyPolicy/773391e6c583b5ff2a5d0746351f3fed/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 14:56:21,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 773391e6c583b5ff2a5d0746351f3fed: Waiting for close lock at 1675954581208Running coprocessor pre-close hooks at 1675954581208Disabling compacts and flushes for region at 1675954581208Disabling writes for close at 1675954581208Writing region close event to WAL at 1675954581209 (+1 ms)Running coprocessor post-close hooks at 1675954581211 (+2 ms)Closed at 1675954581211 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1675954456853.773391e6c583b5ff2a5d0746351f3fed. 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 431c1f7d6260ecba176dba7e512abdb1, disabling compactions & flushes 2023-02-09 14:56:21,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. after waiting 0 ms 2023-02-09 14:56:21,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/431c1f7d6260ecba176dba7e512abdb1/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 14:56:21,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 431c1f7d6260ecba176dba7e512abdb1: Waiting for close lock at 1675954581211Running coprocessor pre-close hooks at 1675954581211Disabling compacts and flushes for region at 1675954581211Disabling writes for close at 1675954581211Writing region close event to WAL at 1675954581212 (+1 ms)Running coprocessor post-close hooks at 1675954581214 (+2 ms)Closed at 1675954581214 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675954468388.431c1f7d6260ecba176dba7e512abdb1. 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 323cead325f2feb2545ff228bca36c06, disabling compactions & flushes 2023-02-09 14:56:21,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. after waiting 0 ms 2023-02-09 14:56:21,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:21,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/323cead325f2feb2545ff228bca36c06/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 323cead325f2feb2545ff228bca36c06: Waiting for close lock at 1675954581214Running coprocessor pre-close hooks at 1675954581214Disabling compacts and flushes for region at 1675954581214Disabling writes for close at 1675954581214Writing region close event to WAL at 1675954581215 (+1 ms)Running coprocessor post-close hooks at 1675954581217 (+2 ms)Closed at 1675954581217 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1675954560707.323cead325f2feb2545ff228bca36c06. 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b989787f43967a90c0c5150dc576648d, disabling compactions & flushes 2023-02-09 14:56:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. after waiting 0 ms 2023-02-09 14:56:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:56:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testScannerOnBulkLoadRefHFiles/b989787f43967a90c0c5150dc576648d/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 14:56:21,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:56:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b989787f43967a90c0c5150dc576648d: Waiting for close lock at 1675954581218Running coprocessor pre-close hooks at 1675954581218Disabling compacts and flushes for region at 1675954581218Disabling writes for close at 1675954581218Writing region close event to WAL at 1675954581218Running coprocessor post-close hooks at 1675954581220 (+2 ms)Closed at 1675954581220 2023-02-09 14:56:21,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1675954453536.b989787f43967a90c0c5150dc576648d. 2023-02-09 14:56:21,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 482df8fbf041dd5fc164a0c315c9e3ed, disabling compactions & flushes 2023-02-09 14:56:21,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:56:21,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:56:21,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. after waiting 0 ms 2023-02-09 14:56:21,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:56:21,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/482df8fbf041dd5fc164a0c315c9e3ed/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 14:56:21,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:56:21,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 482df8fbf041dd5fc164a0c315c9e3ed: Waiting for close lock at 1675954581221Running coprocessor pre-close hooks at 1675954581221Disabling compacts and flushes for region at 1675954581221Disabling writes for close at 1675954581221Writing region close event to WAL at 1675954581221Running coprocessor post-close hooks at 1675954581223 (+2 ms)Closed at 1675954581223 2023-02-09 14:56:21,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675954420504.482df8fbf041dd5fc164a0c315c9e3ed. 2023-02-09 14:56:21,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 932a93e1be1254f02a4c55d7755b4abb, disabling compactions & flushes 2023-02-09 14:56:21,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:21,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:21,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. after waiting 0 ms 2023-02-09 14:56:21,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:21,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/default/testMajorCompactionFromAdmin/932a93e1be1254f02a4c55d7755b4abb/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 14:56:21,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:21,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 932a93e1be1254f02a4c55d7755b4abb: Waiting for close lock at 1675954581224Running coprocessor pre-close hooks at 1675954581224Disabling compacts and flushes for region at 1675954581224Disabling writes for close at 1675954581224Writing region close event to WAL at 1675954581225 (+1 ms)Running coprocessor post-close hooks at 1675954581227 (+2 ms)Closed at 1675954581227 2023-02-09 14:56:21,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1675954560707.932a93e1be1254f02a4c55d7755b4abb. 2023-02-09 14:56:21,335 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 14:56:21,535 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 14:56:21,555 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/rep_barrier/2243cf60302d4f84b4656c19ea8c43fd 2023-02-09 14:56:21,560 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2243cf60302d4f84b4656c19ea8c43fd 2023-02-09 14:56:21,735 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 14:56:21,735 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 14:56:21,735 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 14:56:21,889 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:56:21,936 DEBUG [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 14:56:21,969 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/table/276079bbbc874470b5e6e48f6e1edb88 2023-02-09 14:56:21,973 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 276079bbbc874470b5e6e48f6e1edb88 2023-02-09 14:56:21,974 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/info/4895082d295942089198f9a4c2f3ceb9 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/info/4895082d295942089198f9a4c2f3ceb9 2023-02-09 14:56:21,977 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4895082d295942089198f9a4c2f3ceb9 2023-02-09 14:56:21,977 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/info/4895082d295942089198f9a4c2f3ceb9, entries=226, sequenceid=130, filesize=37.9 K 2023-02-09 14:56:21,978 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/rep_barrier/2243cf60302d4f84b4656c19ea8c43fd as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/rep_barrier/2243cf60302d4f84b4656c19ea8c43fd 2023-02-09 14:56:21,981 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2243cf60302d4f84b4656c19ea8c43fd 2023-02-09 14:56:21,981 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/rep_barrier/2243cf60302d4f84b4656c19ea8c43fd, entries=1, sequenceid=130, filesize=5.0 K 2023-02-09 14:56:21,981 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/.tmp/table/276079bbbc874470b5e6e48f6e1edb88 as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/table/276079bbbc874470b5e6e48f6e1edb88 2023-02-09 14:56:21,984 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 276079bbbc874470b5e6e48f6e1edb88 2023-02-09 14:56:21,984 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/table/276079bbbc874470b5e6e48f6e1edb88, entries=27, sequenceid=130, filesize=6.6 K 2023-02-09 14:56:21,986 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1253ms, sequenceid=130, compaction requested=false 2023-02-09 14:56:21,990 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-09 14:56:21,990 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 14:56:21,990 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 14:56:21,990 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675954580733Running coprocessor pre-close hooks at 1675954580733Disabling compacts and flushes for region at 1675954580733Disabling writes for close at 1675954580733Obtaining lock to block concurrent updates at 1675954580734 (+1 ms)Preparing flush snapshotting stores in 1588230740 at 1675954580734Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1675954580734Flushing stores of hbase:meta,,1.1588230740 at 1675954580734Flushing 1588230740/info: creating writer at 1675954580734Flushing 1588230740/info: appending metadata at 1675954580738 (+4 ms)Flushing 1588230740/info: closing flushed file at 1675954580738Flushing 1588230740/rep_barrier: creating writer at 1675954581149 (+411 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675954581151 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675954581151Flushing 1588230740/table: creating writer at 1675954581560 (+409 ms)Flushing 1588230740/table: appending metadata at 1675954581562 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675954581562Flushing 1588230740/info: reopening flushed file at 1675954581974 (+412 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675954581978 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675954581982 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1253ms, sequenceid=130, compaction requested=false at 1675954581986 (+4 ms)Writing region close event to WAL at 1675954581987 (+1 ms)Running coprocessor post-close hooks at 1675954581990 (+3 ms)Closed at 1675954581990 2023-02-09 14:56:21,990 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 14:56:22,136 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43455,1675954416872; all regions closed. 2023-02-09 14:56:22,143 DEBUG [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/oldWALs 2023-02-09 14:56:22,143 INFO [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43455%2C1675954416872.meta:.meta(num 1675954418716) 2023-02-09 14:56:22,147 DEBUG [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/oldWALs 2023-02-09 14:56:22,147 INFO [RS:0;jenkins-hbase4:43455] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43455%2C1675954416872:(num 1675954418189) 2023-02-09 14:56:22,147 DEBUG [RS:0;jenkins-hbase4:43455] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:22,147 INFO [RS:0;jenkins-hbase4:43455] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:56:22,147 INFO [RS:0;jenkins-hbase4:43455] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 14:56:22,147 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:56:22,148 INFO [RS:0;jenkins-hbase4:43455] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43455 2023-02-09 14:56:22,157 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:56:22,157 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43455,1675954416872 2023-02-09 14:56:22,157 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:56:22,157 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43455,1675954416872] 2023-02-09 14:56:22,158 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43455,1675954416872; numProcessing=1 2023-02-09 14:56:22,159 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43455,1675954416872 already deleted, retry=false 2023-02-09 14:56:22,159 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43455,1675954416872 expired; onlineServers=0 2023-02-09 14:56:22,159 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38213,1675954415718' ***** 2023-02-09 14:56:22,159 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 14:56:22,160 DEBUG [M:0;jenkins-hbase4:38213] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fb60ee8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:56:22,160 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38213,1675954415718 2023-02-09 14:56:22,160 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38213,1675954415718; all regions closed. 2023-02-09 14:56:22,160 DEBUG [M:0;jenkins-hbase4:38213] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:56:22,160 DEBUG [M:0;jenkins-hbase4:38213] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 14:56:22,160 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 14:56:22,160 DEBUG [M:0;jenkins-hbase4:38213] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 14:56:22,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954417864] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954417864,5,FailOnTimeoutGroup] 2023-02-09 14:56:22,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954417864] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954417864,5,FailOnTimeoutGroup] 2023-02-09 14:56:22,160 INFO [M:0;jenkins-hbase4:38213] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 14:56:22,160 INFO [M:0;jenkins-hbase4:38213] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 14:56:22,160 INFO [M:0;jenkins-hbase4:38213] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 14:56:22,161 DEBUG [M:0;jenkins-hbase4:38213] master.HMaster(1502): Stopping service threads 2023-02-09 14:56:22,161 INFO [M:0;jenkins-hbase4:38213] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 14:56:22,161 ERROR [M:0;jenkins-hbase4:38213] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-09 14:56:22,162 INFO [M:0;jenkins-hbase4:38213] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:56:22,162 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 14:56:22,163 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 14:56:22,163 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:56:22,163 DEBUG [M:0;jenkins-hbase4:38213] zookeeper.ZKUtil(398): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 14:56:22,163 WARN [M:0;jenkins-hbase4:38213] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 14:56:22,163 INFO [M:0;jenkins-hbase4:38213] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:56:22,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:56:22,163 INFO [M:0;jenkins-hbase4:38213] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 14:56:22,163 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 14:56:22,163 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:56:22,163 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:56:22,163 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 14:56:22,163 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:56:22,163 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.72 KB heapSize=598.85 KB 2023-02-09 14:56:22,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:56:22,259 INFO [RS:0;jenkins-hbase4:43455] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43455,1675954416872; zookeeper connection closed. 2023-02-09 14:56:22,259 ERROR [Listener at localhost/33189-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4193957 rejected from java.util.concurrent.ThreadPoolExecutor@26bd0c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 534] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:56:22,259 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@65bb1f70] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@65bb1f70 2023-02-09 14:56:22,259 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): regionserver:43455-0x1003992ceed0001, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:56:22,259 ERROR [Listener at localhost/33189-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@39356082 rejected from java.util.concurrent.ThreadPoolExecutor@26bd0c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 534] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:56:22,259 INFO [Listener at localhost/33189] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-09 14:56:22,571 INFO [M:0;jenkins-hbase4:38213] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.72 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e51461ed6acb4a2aa5be0217e95a4a0e 2023-02-09 14:56:22,574 INFO [M:0;jenkins-hbase4:38213] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e51461ed6acb4a2aa5be0217e95a4a0e 2023-02-09 14:56:22,575 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/e51461ed6acb4a2aa5be0217e95a4a0e as hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e51461ed6acb4a2aa5be0217e95a4a0e 2023-02-09 14:56:22,578 INFO [M:0;jenkins-hbase4:38213] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e51461ed6acb4a2aa5be0217e95a4a0e 2023-02-09 14:56:22,578 INFO [M:0;jenkins-hbase4:38213] regionserver.HStore(1080): Added hdfs://localhost:44771/user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/e51461ed6acb4a2aa5be0217e95a4a0e, entries=123, sequenceid=1011, filesize=33.3 K 2023-02-09 14:56:22,578 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegion(2947): Finished flush of dataSize ~512.72 KB/525030, heapSize ~598.84 KB/613208, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1011, compaction requested=false 2023-02-09 14:56:22,579 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:56:22,579 DEBUG [M:0;jenkins-hbase4:38213] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675954582163Disabling compacts and flushes for region at 1675954582163Disabling writes for close at 1675954582163Obtaining lock to block concurrent updates at 1675954582163Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675954582163Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525030, getHeapSize=613208, getOffHeapSize=0, getCellsCount=1166 at 1675954582163Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675954582164 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675954582164Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675954582167 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675954582167Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675954582575 (+408 ms)Finished flush of dataSize ~512.72 KB/525030, heapSize ~598.84 KB/613208, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1011, compaction requested=false at 1675954582578 (+3 ms)Writing region close event to WAL at 1675954582579 (+1 ms)Closed at 1675954582579 2023-02-09 14:56:22,581 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/d1829945-e048-b42e-d949-1e562615af7d/MasterData/WALs/jenkins-hbase4.apache.org,38213,1675954415718/jenkins-hbase4.apache.org%2C38213%2C1675954415718.1675954417284 not finished, retry = 0 2023-02-09 14:56:22,682 INFO [M:0;jenkins-hbase4:38213] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 14:56:22,682 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:56:22,683 INFO [M:0;jenkins-hbase4:38213] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38213 2023-02-09 14:56:22,685 DEBUG [M:0;jenkins-hbase4:38213] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38213,1675954415718 already deleted, retry=false 2023-02-09 14:56:22,787 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:56:22,787 INFO [M:0;jenkins-hbase4:38213] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38213,1675954415718; zookeeper connection closed. 2023-02-09 14:56:22,787 ERROR [Listener at localhost/33189-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@182cb565 rejected from java.util.concurrent.ThreadPoolExecutor@1d9f9790[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:56:22,787 DEBUG [Listener at localhost/33189-EventThread] zookeeper.ZKWatcher(600): master:38213-0x1003992ceed0000, quorum=127.0.0.1:56347, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:56:22,788 ERROR [Listener at localhost/33189-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1aa34a21 rejected from java.util.concurrent.ThreadPoolExecutor@1d9f9790[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:56:22,789 WARN [Listener at localhost/33189] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 14:56:22,793 INFO [Listener at localhost/33189] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:56:22,899 WARN [BP-477111177-172.31.14.131-1675954412651 heartbeating to localhost/127.0.0.1:44771] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 14:56:22,901 WARN [BP-477111177-172.31.14.131-1675954412651 heartbeating to localhost/127.0.0.1:44771] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-477111177-172.31.14.131-1675954412651 (Datanode Uuid e1049c1d-88a3-49fc-8224-542a4e55a7fe) service to localhost/127.0.0.1:44771 2023-02-09 14:56:22,904 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/dfs/data/data1/current/BP-477111177-172.31.14.131-1675954412651] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:56:22,904 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/07d43c8f-afbb-7a10-cdca-9e60011e04b6/cluster_e41dbc4b-7bb3-2250-6130-2c5db1cbb877/dfs/data/data2/current/BP-477111177-172.31.14.131-1675954412651] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:56:22,932 INFO [Listener at localhost/33189] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:56:23,075 INFO [Listener at localhost/33189] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 14:56:23,164 INFO [Listener at localhost/33189] hbase.HBaseTestingUtility(1293): Minicluster is down