2023-02-09 22:53:29,338 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-09 22:53:29,597 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0 2023-02-09 22:53:29,612 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 22:53:29,612 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade, deleteOnExit=true 2023-02-09 22:53:29,612 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 22:53:29,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/test.cache.data in system properties and HBase conf 2023-02-09 22:53:29,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 22:53:29,613 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/hadoop.log.dir in system properties and HBase conf 2023-02-09 22:53:29,614 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 22:53:29,614 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 22:53:29,614 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 22:53:29,727 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 22:53:30,132 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 22:53:30,136 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 22:53:30,137 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 22:53:30,137 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 22:53:30,137 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 22:53:30,138 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 22:53:30,138 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 22:53:30,138 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 22:53:30,139 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 22:53:30,139 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 22:53:30,139 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/nfs.dump.dir in system properties and HBase conf 2023-02-09 22:53:30,140 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/java.io.tmpdir in system properties and HBase conf 2023-02-09 22:53:30,140 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 22:53:30,141 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 22:53:30,141 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 22:53:30,659 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 22:53:30,663 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 22:53:30,999 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 22:53:31,190 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 22:53:31,211 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:31,250 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:31,284 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/java.io.tmpdir/Jetty_localhost_37951_hdfs____yv85ej/webapp 2023-02-09 22:53:31,437 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37951 2023-02-09 22:53:31,455 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 22:53:31,455 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 22:53:32,027 WARN [Listener at localhost/33985] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:32,088 WARN [Listener at localhost/33985] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 22:53:32,106 WARN [Listener at localhost/33985] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 22:53:32,113 INFO [Listener at localhost/33985] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 22:53:32,118 INFO [Listener at localhost/33985] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/java.io.tmpdir/Jetty_localhost_34359_datanode____hc3tvh/webapp 2023-02-09 22:53:32,213 INFO [Listener at localhost/33985] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34359 2023-02-09 22:53:32,494 WARN [Listener at localhost/33819] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 22:53:32,901 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa94345627433b18e: Processing first storage report for DS-6b2d2ece-a843-4af0-8616-feb91802fda7 from datanode 15ba31fb-e227-4287-a469-dff6dd827365 2023-02-09 22:53:32,902 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa94345627433b18e: from storage DS-6b2d2ece-a843-4af0-8616-feb91802fda7 node DatanodeRegistration(127.0.0.1:34379, datanodeUuid=15ba31fb-e227-4287-a469-dff6dd827365, infoPort=46561, infoSecurePort=0, ipcPort=33819, storageInfo=lv=-57;cid=testClusterID;nsid=1498865328;c=1675983210759), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 22:53:32,902 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa94345627433b18e: Processing first storage report for DS-44158523-aed8-40a1-84e4-77dad69067f3 from datanode 15ba31fb-e227-4287-a469-dff6dd827365 2023-02-09 22:53:32,902 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa94345627433b18e: from storage DS-44158523-aed8-40a1-84e4-77dad69067f3 node DatanodeRegistration(127.0.0.1:34379, datanodeUuid=15ba31fb-e227-4287-a469-dff6dd827365, infoPort=46561, infoSecurePort=0, ipcPort=33819, storageInfo=lv=-57;cid=testClusterID;nsid=1498865328;c=1675983210759), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 22:53:32,988 DEBUG [Listener at localhost/33819] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0 2023-02-09 22:53:33,048 INFO [Listener at localhost/33819] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/zookeeper_0, clientPort=59035, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 22:53:33,063 INFO [Listener at localhost/33819] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=59035 2023-02-09 22:53:33,070 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:33,072 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:33,725 INFO [Listener at localhost/33819] util.FSUtils(479): Created version file at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160 with version=8 2023-02-09 22:53:33,725 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/hbase-staging 2023-02-09 22:53:34,051 INFO [Listener at localhost/33819] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 22:53:34,505 INFO [Listener at localhost/33819] client.ConnectionUtils(127): master/jenkins-hbase9:0 server-side Connection retries=3 2023-02-09 22:53:34,536 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:34,536 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:34,536 INFO [Listener at localhost/33819] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:53:34,537 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:34,537 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:53:34,674 INFO [Listener at localhost/33819] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:53:34,747 DEBUG [Listener at localhost/33819] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 22:53:34,839 INFO [Listener at localhost/33819] ipc.NettyRpcServer(120): Bind to /172.31.2.10:35735 2023-02-09 22:53:34,849 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:34,851 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:34,871 INFO [Listener at localhost/33819] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35735 connecting to ZooKeeper ensemble=127.0.0.1:59035 2023-02-09 22:53:34,910 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:357350x0, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:53:34,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:35735-0x1003b4b31d80000 connected 2023-02-09 22:53:34,941 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:53:34,942 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:53:34,945 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:53:34,952 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35735 2023-02-09 22:53:34,953 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35735 2023-02-09 22:53:34,953 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35735 2023-02-09 22:53:34,953 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35735 2023-02-09 22:53:34,953 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35735 2023-02-09 22:53:34,959 INFO [Listener at localhost/33819] master.HMaster(439): hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160, hbase.cluster.distributed=false 2023-02-09 22:53:35,027 INFO [Listener at localhost/33819] client.ConnectionUtils(127): regionserver/jenkins-hbase9:0 server-side Connection retries=3 2023-02-09 22:53:35,027 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:35,027 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:35,028 INFO [Listener at localhost/33819] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 22:53:35,028 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 22:53:35,028 INFO [Listener at localhost/33819] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 22:53:35,033 INFO [Listener at localhost/33819] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 22:53:35,036 INFO [Listener at localhost/33819] ipc.NettyRpcServer(120): Bind to /172.31.2.10:39093 2023-02-09 22:53:35,038 INFO [Listener at localhost/33819] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 22:53:35,044 DEBUG [Listener at localhost/33819] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 22:53:35,045 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:35,047 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:35,049 INFO [Listener at localhost/33819] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39093 connecting to ZooKeeper ensemble=127.0.0.1:59035 2023-02-09 22:53:35,052 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:390930x0, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 22:53:35,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39093-0x1003b4b31d80001 connected 2023-02-09 22:53:35,054 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:53:35,055 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:53:35,056 DEBUG [Listener at localhost/33819] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 22:53:35,057 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39093 2023-02-09 22:53:35,057 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39093 2023-02-09 22:53:35,057 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39093 2023-02-09 22:53:35,058 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39093 2023-02-09 22:53:35,058 DEBUG [Listener at localhost/33819] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39093 2023-02-09 22:53:35,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:35,069 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 22:53:35,070 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:35,087 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:53:35,087 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 22:53:35,087 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:35,088 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 22:53:35,090 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase9.apache.org,35735,1675983213878 from backup master directory 2023-02-09 22:53:35,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 22:53:35,093 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:35,094 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 22:53:35,094 WARN [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:53:35,094 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:35,097 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 22:53:35,098 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 22:53:35,581 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/hbase.id with ID: 8d6f738d-a584-4e95-95d9-dbb1dafaaae5 2023-02-09 22:53:35,622 INFO [master/jenkins-hbase9:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:35,636 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:36,073 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x613b4cee to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:36,105 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62730106, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:36,128 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 22:53:36,130 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 22:53:36,146 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 22:53:36,146 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 22:53:36,148 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:53:36,152 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:53:36,153 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:53:36,185 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store-tmp 2023-02-09 22:53:36,212 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:36,213 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 22:53:36,213 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:53:36,213 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:53:36,213 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 22:53:36,213 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:53:36,213 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:53:36,214 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675983216213Disabling compacts and flushes for region at 1675983216213Disabling writes for close at 1675983216213Writing region close event to WAL at 1675983216213Closed at 1675983216213 2023-02-09 22:53:36,215 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/WALs/jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:36,234 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C35735%2C1675983213878, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/WALs/jenkins-hbase9.apache.org,35735,1675983213878, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/oldWALs, maxLogs=10 2023-02-09 22:53:36,284 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK] 2023-02-09 22:53:36,291 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 22:53:36,351 INFO [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/WALs/jenkins-hbase9.apache.org,35735,1675983213878/jenkins-hbase9.apache.org%2C35735%2C1675983213878.1675983216244 2023-02-09 22:53:36,352 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK]] 2023-02-09 22:53:36,352 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:36,353 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:36,356 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,357 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,410 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,417 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 22:53:36,439 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 22:53:36,450 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:36,456 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,458 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,474 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 22:53:36,479 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:53:36,480 INFO [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60170883, jitterRate=-0.10338397324085236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:53:36,480 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675983216382Initializing all the Stores at 1675983216384 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983216384Cleaning up temporary data from old regions at 1675983216466 (+82 ms)Cleaning up detritus from prior splits at 1675983216467 (+1 ms)Region opened successfully at 1675983216480 (+13 ms) 2023-02-09 22:53:36,481 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 22:53:36,501 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-09 22:53:36,501 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-09 22:53:36,503 INFO [master/jenkins-hbase9:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 22:53:36,505 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 22:53:36,535 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-09 22:53:36,536 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 22:53:36,560 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 22:53:36,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 22:53:36,588 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 22:53:36,592 INFO [master/jenkins-hbase9:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 22:53:36,594 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 22:53:36,599 INFO [master/jenkins-hbase9:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 22:53:36,603 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 22:53:36,606 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:36,607 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 22:53:36,608 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 22:53:36,619 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 22:53:36,624 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:53:36,624 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 22:53:36,624 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:36,625 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase9.apache.org,35735,1675983213878, sessionid=0x1003b4b31d80000, setting cluster-up flag (Was=false) 2023-02-09 22:53:36,639 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:36,645 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 22:53:36,647 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:36,652 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:36,658 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 22:53:36,660 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:36,662 WARN [master/jenkins-hbase9:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp 2023-02-09 22:53:36,763 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(952): ClusterId : 8d6f738d-a584-4e95-95d9-dbb1dafaaae5 2023-02-09 22:53:36,767 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 22:53:36,772 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 22:53:36,772 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 22:53:36,776 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 22:53:36,776 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 22:53:36,777 DEBUG [RS:0;jenkins-hbase9:39093] zookeeper.ReadOnlyZKClient(139): Connect 0x50311e2c to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:36,782 DEBUG [RS:0;jenkins-hbase9:39093] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ef6ea5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:36,782 DEBUG [RS:0;jenkins-hbase9:39093] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ac9755c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:53:36,788 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=5, maxPoolSize=5 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase9:0, corePoolSize=10, maxPoolSize=10 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:53:36,789 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,791 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675983246791 2023-02-09 22:53:36,792 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 22:53:36,798 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 22:53:36,798 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 22:53:36,803 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 22:53:36,804 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 22:53:36,807 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase9:39093 2023-02-09 22:53:36,810 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 22:53:36,811 INFO [RS:0;jenkins-hbase9:39093] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 22:53:36,811 INFO [RS:0;jenkins-hbase9:39093] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 22:53:36,811 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 22:53:36,811 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 22:53:36,811 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 22:53:36,811 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 22:53:36,813 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase9.apache.org,35735,1675983213878 with isa=jenkins-hbase9.apache.org/172.31.2.10:39093, startcode=1675983215026 2023-02-09 22:53:36,815 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,816 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 22:53:36,818 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 22:53:36,818 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 22:53:36,821 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 22:53:36,822 INFO [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 22:53:36,826 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983216825,5,FailOnTimeoutGroup] 2023-02-09 22:53:36,827 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983216827,5,FailOnTimeoutGroup] 2023-02-09 22:53:36,827 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,827 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 22:53:36,829 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,830 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,836 DEBUG [RS:0;jenkins-hbase9:39093] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 22:53:36,882 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45273, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 22:53:36,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.ServerManager(394): Registering regionserver=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:36,907 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160 2023-02-09 22:53:36,907 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33985 2023-02-09 22:53:36,907 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 22:53:36,912 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:53:36,912 DEBUG [RS:0;jenkins-hbase9:39093] zookeeper.ZKUtil(162): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:36,913 WARN [RS:0;jenkins-hbase9:39093] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 22:53:36,913 INFO [RS:0;jenkins-hbase9:39093] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:53:36,913 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/WALs/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:36,914 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase9.apache.org,39093,1675983215026] 2023-02-09 22:53:36,922 DEBUG [RS:0;jenkins-hbase9:39093] zookeeper.ZKUtil(162): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:36,931 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 22:53:36,939 INFO [RS:0;jenkins-hbase9:39093] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 22:53:36,959 INFO [RS:0;jenkins-hbase9:39093] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 22:53:36,964 INFO [RS:0;jenkins-hbase9:39093] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:53:36,964 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,965 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 22:53:36,973 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,973 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,973 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,973 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase9:0, corePoolSize=2, maxPoolSize=2 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,974 DEBUG [RS:0;jenkins-hbase9:39093] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase9:0, corePoolSize=1, maxPoolSize=1 2023-02-09 22:53:36,975 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,975 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,975 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:36,990 INFO [RS:0;jenkins-hbase9:39093] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 22:53:36,992 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,39093,1675983215026-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:37,007 INFO [RS:0;jenkins-hbase9:39093] regionserver.Replication(203): jenkins-hbase9.apache.org,39093,1675983215026 started 2023-02-09 22:53:37,008 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1638): Serving as jenkins-hbase9.apache.org,39093,1675983215026, RpcServer on jenkins-hbase9.apache.org/172.31.2.10:39093, sessionid=0x1003b4b31d80001 2023-02-09 22:53:37,008 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 22:53:37,008 DEBUG [RS:0;jenkins-hbase9:39093] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:37,008 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,39093,1675983215026' 2023-02-09 22:53:37,008 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase9.apache.org,39093,1675983215026' 2023-02-09 22:53:37,009 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:53:37,010 DEBUG [RS:0;jenkins-hbase9:39093] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:53:37,010 DEBUG [RS:0;jenkins-hbase9:39093] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 22:53:37,010 INFO [RS:0;jenkins-hbase9:39093] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 22:53:37,010 INFO [RS:0;jenkins-hbase9:39093] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 22:53:37,121 INFO [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C39093%2C1675983215026, suffix=, logDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/WALs/jenkins-hbase9.apache.org,39093,1675983215026, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/oldWALs, maxLogs=32 2023-02-09 22:53:37,139 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK] 2023-02-09 22:53:37,142 INFO [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/WALs/jenkins-hbase9.apache.org,39093,1675983215026/jenkins-hbase9.apache.org%2C39093%2C1675983215026.1675983217124 2023-02-09 22:53:37,143 DEBUG [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK]] 2023-02-09 22:53:37,234 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:37,235 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:37,235 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160 2023-02-09 22:53:37,648 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:37,651 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 22:53:37,654 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/info 2023-02-09 22:53:37,655 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 22:53:37,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:37,656 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 22:53:37,659 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:53:37,659 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 22:53:37,660 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:37,660 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 22:53:37,662 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/table 2023-02-09 22:53:37,663 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 22:53:37,664 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:37,665 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740 2023-02-09 22:53:37,666 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740 2023-02-09 22:53:37,670 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 22:53:37,671 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 22:53:37,676 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:53:37,677 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63569492, jitterRate=-0.05274075269699097}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 22:53:37,677 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675983217649Initializing all the Stores at 1675983217650 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983217650Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983217650Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983217650Cleaning up temporary data from old regions at 1675983217668 (+18 ms)Cleaning up detritus from prior splits at 1675983217668Region opened successfully at 1675983217677 (+9 ms) 2023-02-09 22:53:37,677 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 22:53:37,677 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 22:53:37,677 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 22:53:37,677 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 22:53:37,677 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 22:53:37,678 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 22:53:37,678 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675983217677Disabling compacts and flushes for region at 1675983217677Disabling writes for close at 1675983217677Writing region close event to WAL at 1675983217678 (+1 ms)Closed at 1675983217678 2023-02-09 22:53:37,684 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 22:53:37,684 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 22:53:37,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 22:53:37,703 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 22:53:37,705 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 22:53:37,857 DEBUG [jenkins-hbase9:35735] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-09 22:53:37,860 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,39093,1675983215026, state=OPENING 2023-02-09 22:53:37,867 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 22:53:37,870 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:37,870 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 22:53:37,873 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:38,056 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:38,058 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:53:38,061 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53038, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:53:38,072 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 22:53:38,073 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 22:53:38,076 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase9.apache.org%2C39093%2C1675983215026.meta, suffix=.meta, logDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/WALs/jenkins-hbase9.apache.org,39093,1675983215026, archiveDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/oldWALs, maxLogs=32 2023-02-09 22:53:38,092 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK] 2023-02-09 22:53:38,097 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/WALs/jenkins-hbase9.apache.org,39093,1675983215026/jenkins-hbase9.apache.org%2C39093%2C1675983215026.meta.1675983218077.meta 2023-02-09 22:53:38,099 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34379,DS-6b2d2ece-a843-4af0-8616-feb91802fda7,DISK]] 2023-02-09 22:53:38,099 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:38,101 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 22:53:38,116 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 22:53:38,121 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 22:53:38,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 22:53:38,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:38,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 22:53:38,126 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 22:53:38,129 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 22:53:38,131 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/info 2023-02-09 22:53:38,131 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/info 2023-02-09 22:53:38,131 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 22:53:38,132 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:38,132 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 22:53:38,133 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:53:38,134 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/rep_barrier 2023-02-09 22:53:38,134 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 22:53:38,135 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:38,135 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 22:53:38,137 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/table 2023-02-09 22:53:38,137 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/table 2023-02-09 22:53:38,138 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 22:53:38,138 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:38,140 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740 2023-02-09 22:53:38,142 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740 2023-02-09 22:53:38,146 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 22:53:38,148 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 22:53:38,149 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64084175, jitterRate=-0.045071378350257874}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 22:53:38,149 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675983218127Writing region info on filesystem at 1675983218127Initializing all the Stores at 1675983218128 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983218129 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675983218129Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983218129Cleaning up temporary data from old regions at 1675983218144 (+15 ms)Cleaning up detritus from prior splits at 1675983218144Running coprocessor post-open hooks at 1675983218149 (+5 ms)Region opened successfully at 1675983218149 2023-02-09 22:53:38,158 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675983218048 2023-02-09 22:53:38,175 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 22:53:38,176 INFO [RS_OPEN_META-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 22:53:38,176 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase9.apache.org,39093,1675983215026, state=OPEN 2023-02-09 22:53:38,179 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 22:53:38,179 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 22:53:38,183 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 22:53:38,184 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase9.apache.org,39093,1675983215026 in 306 msec 2023-02-09 22:53:38,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 22:53:38,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 493 msec 2023-02-09 22:53:38,196 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.4880 sec 2023-02-09 22:53:38,196 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675983215084, completionTime=-1 2023-02-09 22:53:38,197 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-09 22:53:38,197 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 22:53:38,261 DEBUG [hconnection-0x3138c7e0-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:53:38,264 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53042, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:53:38,280 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-09 22:53:38,281 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675983278281 2023-02-09 22:53:38,281 INFO [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675983338281 2023-02-09 22:53:38,281 INFO [master/jenkins-hbase9:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 83 msec 2023-02-09 22:53:38,303 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,35735,1675983213878-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:38,304 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,35735,1675983213878-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:38,304 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,35735,1675983213878-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:38,306 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase9:35735, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:38,306 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 22:53:38,311 DEBUG [master/jenkins-hbase9:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 22:53:38,319 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 22:53:38,320 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 22:53:38,329 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 22:53:38,331 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:53:38,334 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:53:38,353 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:38,355 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3 empty. 2023-02-09 22:53:38,356 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:38,356 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 22:53:38,376 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:38,378 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => b772f3a0c39f9eb94af543c4117d03e3, NAME => 'hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing b772f3a0c39f9eb94af543c4117d03e3, disabling compactions & flushes 2023-02-09 22:53:38,790 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. after waiting 0 ms 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:38,790 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:38,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for b772f3a0c39f9eb94af543c4117d03e3: Waiting for close lock at 1675983218790Disabling compacts and flushes for region at 1675983218790Disabling writes for close at 1675983218790Writing region close event to WAL at 1675983218790Closed at 1675983218790 2023-02-09 22:53:38,795 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:53:38,811 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983218798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983218798"}]},"ts":"1675983218798"} 2023-02-09 22:53:38,837 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:53:38,838 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:53:38,842 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983218839"}]},"ts":"1675983218839"} 2023-02-09 22:53:38,845 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 22:53:38,857 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b772f3a0c39f9eb94af543c4117d03e3, ASSIGN}] 2023-02-09 22:53:38,859 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b772f3a0c39f9eb94af543c4117d03e3, ASSIGN 2023-02-09 22:53:38,861 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b772f3a0c39f9eb94af543c4117d03e3, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:53:39,012 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b772f3a0c39f9eb94af543c4117d03e3, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:39,012 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983219011"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983219011"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983219011"}]},"ts":"1675983219011"} 2023-02-09 22:53:39,017 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b772f3a0c39f9eb94af543c4117d03e3, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:39,177 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:39,177 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b772f3a0c39f9eb94af543c4117d03e3, NAME => 'hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:39,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:39,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,178 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,180 INFO [StoreOpener-b772f3a0c39f9eb94af543c4117d03e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,182 DEBUG [StoreOpener-b772f3a0c39f9eb94af543c4117d03e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/info 2023-02-09 22:53:39,182 DEBUG [StoreOpener-b772f3a0c39f9eb94af543c4117d03e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/info 2023-02-09 22:53:39,183 INFO [StoreOpener-b772f3a0c39f9eb94af543c4117d03e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b772f3a0c39f9eb94af543c4117d03e3 columnFamilyName info 2023-02-09 22:53:39,183 INFO [StoreOpener-b772f3a0c39f9eb94af543c4117d03e3-1] regionserver.HStore(310): Store=b772f3a0c39f9eb94af543c4117d03e3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:39,185 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,186 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,190 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:53:39,193 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:53:39,194 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened b772f3a0c39f9eb94af543c4117d03e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71695640, jitterRate=0.06834828853607178}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:53:39,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for b772f3a0c39f9eb94af543c4117d03e3: Running coprocessor pre-open hook at 1675983219178Writing region info on filesystem at 1675983219178Initializing all the Stores at 1675983219180 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675983219180Cleaning up temporary data from old regions at 1675983219186 (+6 ms)Cleaning up detritus from prior splits at 1675983219187 (+1 ms)Running coprocessor post-open hooks at 1675983219194 (+7 ms)Region opened successfully at 1675983219194 2023-02-09 22:53:39,196 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3., pid=6, masterSystemTime=1675983219170 2023-02-09 22:53:39,200 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:39,200 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:53:39,201 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b772f3a0c39f9eb94af543c4117d03e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:39,202 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675983219201"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983219201"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983219201"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983219201"}]},"ts":"1675983219201"} 2023-02-09 22:53:39,209 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 22:53:39,209 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b772f3a0c39f9eb94af543c4117d03e3, server=jenkins-hbase9.apache.org,39093,1675983215026 in 190 msec 2023-02-09 22:53:39,212 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 22:53:39,212 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b772f3a0c39f9eb94af543c4117d03e3, ASSIGN in 352 msec 2023-02-09 22:53:39,213 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:53:39,214 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983219214"}]},"ts":"1675983219214"} 2023-02-09 22:53:39,216 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 22:53:39,220 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:53:39,223 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 899 msec 2023-02-09 22:53:39,232 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 22:53:39,234 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:53:39,234 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:53:39,266 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 22:53:39,281 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:53:39,288 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-02-09 22:53:39,301 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 22:53:39,311 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:53:39,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-02-09 22:53:39,328 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 22:53:39,330 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 22:53:39,330 INFO [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 4.236sec 2023-02-09 22:53:39,332 INFO [master/jenkins-hbase9:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 22:53:39,333 INFO [master/jenkins-hbase9:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 22:53:39,333 INFO [master/jenkins-hbase9:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 22:53:39,334 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,35735,1675983213878-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 22:53:39,335 INFO [master/jenkins-hbase9:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase9.apache.org,35735,1675983213878-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 22:53:39,346 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 22:53:39,369 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(139): Connect 0x0672b468 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:39,373 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@561bf69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:39,386 DEBUG [hconnection-0x78f916d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:53:39,396 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53046, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:53:39,406 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:53:39,407 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(139): Connect 0x33fc1e18 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:39,412 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b586bb5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:39,413 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:53:39,433 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=335, ProcessCount=168, AvailableMemoryMB=11776 2023-02-09 22:53:39,443 DEBUG [Listener at localhost/33819] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:53:39,446 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58632, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:53:39,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:53:39,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:39,461 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:53:39,463 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:53:39,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-09 22:53:39,466 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:39,467 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 empty. 2023-02-09 22:53:39,468 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:39,469 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-09 22:53:39,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:39,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:39,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:39,885 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:39,888 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0912312fd88f712f69082600e46ca0c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:53:40,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:40,300 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:40,301 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 0912312fd88f712f69082600e46ca0c8, disabling compactions & flushes 2023-02-09 22:53:40,301 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,301 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,301 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. after waiting 0 ms 2023-02-09 22:53:40,301 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,301 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,302 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 0912312fd88f712f69082600e46ca0c8: Waiting for close lock at 1675983220301Disabling compacts and flushes for region at 1675983220301Disabling writes for close at 1675983220301Writing region close event to WAL at 1675983220301Closed at 1675983220301 2023-02-09 22:53:40,306 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:53:40,308 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983220308"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983220308"}]},"ts":"1675983220308"} 2023-02-09 22:53:40,311 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:53:40,312 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:53:40,313 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983220312"}]},"ts":"1675983220312"} 2023-02-09 22:53:40,315 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-09 22:53:40,318 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, ASSIGN}] 2023-02-09 22:53:40,321 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, ASSIGN 2023-02-09 22:53:40,322 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:53:40,474 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:40,474 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983220473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983220473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983220473"}]},"ts":"1675983220473"} 2023-02-09 22:53:40,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:40,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:40,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0912312fd88f712f69082600e46ca0c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:40,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:40,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,642 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,645 DEBUG [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:40,645 DEBUG [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:40,649 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0912312fd88f712f69082600e46ca0c8 columnFamilyName family1 2023-02-09 22:53:40,653 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] regionserver.HStore(310): Store=0912312fd88f712f69082600e46ca0c8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:40,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:40,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:53:40,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 0912312fd88f712f69082600e46ca0c8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74480804, jitterRate=0.10985046625137329}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:53:40,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 0912312fd88f712f69082600e46ca0c8: Running coprocessor pre-open hook at 1675983220638Writing region info on filesystem at 1675983220638Initializing all the Stores at 1675983220640 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983220640Cleaning up temporary data from old regions at 1675983220657 (+17 ms)Cleaning up detritus from prior splits at 1675983220657Running coprocessor post-open hooks at 1675983220672 (+15 ms)Region opened successfully at 1675983220672 2023-02-09 22:53:40,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., pid=11, masterSystemTime=1675983220631 2023-02-09 22:53:40,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:40,677 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:40,678 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983220677"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983220677"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983220677"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983220677"}]},"ts":"1675983220677"} 2023-02-09 22:53:40,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 22:53:40,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026 in 204 msec 2023-02-09 22:53:40,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 22:53:40,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, ASSIGN in 377 msec 2023-02-09 22:53:40,701 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:53:40,702 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983220702"}]},"ts":"1675983220702"} 2023-02-09 22:53:40,704 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-09 22:53:40,708 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:53:40,711 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2500 sec 2023-02-09 22:53:41,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-09 22:53:41,594 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-09 22:53:41,611 DEBUG [hconnection-0x7aa6bd5c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:53:41,614 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:60684, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:53:41,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:41,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,785 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:41,785 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:41,786 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:41,786 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:41,787 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,787 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,792 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:41,792 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:41,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:41,792 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:41,792 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,793 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:41,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:41,797 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:41,797 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:41,798 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:41,798 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:41,799 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:41,799 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:41,800 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:41,801 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:53:42,357 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ec5257478a334b5fb47701000aa71b75 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec5257478a334b5fb47701000aa71b75 2023-02-09 22:53:42,377 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/05bf8894f1be4ceb99ecfcbc9ed96c59 2023-02-09 22:53:42,396 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/05bf8894f1be4ceb99ecfcbc9ed96c59 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59 2023-02-09 22:53:42,408 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-09 22:53:42,412 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 612ms, sequenceid=5, compaction requested=false 2023-02-09 22:53:42,414 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:42,417 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:42,417 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:42,418 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:42,419 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:42,419 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,419 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:42,419 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:42,421 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,422 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:42,422 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,423 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:42,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:42,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:42,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:42,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:42,425 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:42,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ae44c5c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:42,425 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,426 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,427 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:42,427 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:42,427 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:42,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:42,429 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:42,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:42,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:42,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:42,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:42,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:42,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:42,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:42,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:42,434 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:42,434 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:42,434 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:42,434 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,434 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:42,441 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,441 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:42,441 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:42,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,446 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,455 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,456 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,456 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:42,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:42,457 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:42,457 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:42,457 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:42,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:42,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:42,459 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:42,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:42,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:42,459 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:42,560 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:42,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:42,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:42,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:42,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,588 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:42,588 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:42,589 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:42,589 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:42,590 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,590 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,591 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:42,591 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:42,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:42,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:42,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:42,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:42,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:42,594 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:42,594 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:42,595 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:42,595 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:42,595 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:42,595 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:42,595 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:42,595 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:53:42,936 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 22:53:42,937 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 22:53:43,034 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128888e1a3a132b492a9e291a8bc80c53f3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128888e1a3a132b492a9e291a8bc80c53f3 2023-02-09 22:53:43,452 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/3920e07c91904bed9e88ca0e51bf8577 2023-02-09 22:53:43,463 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/3920e07c91904bed9e88ca0e51bf8577 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577 2023-02-09 22:53:43,472 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:53:43,473 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 878ms, sequenceid=9, compaction requested=false 2023-02-09 22:53:43,474 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:43,474 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:43,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:43,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:43,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:43,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:43,476 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,476 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:43,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:43,476 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:43,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:43,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:43,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:43,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:43,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ee08586[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:43,478 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:43,478 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,480 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,480 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:43,480 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:43,480 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:43,481 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,481 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:43,482 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:43,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:43,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:43,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:43,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:43,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:43,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:43,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:43,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:43,486 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:43,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:43,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:43,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,486 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:43,492 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,492 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:43,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,492 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:43,493 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:43,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:43,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,494 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:43,499 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:43,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:43,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:43,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:43,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:43,499 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:43,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,500 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:43,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:43,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:43,500 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:43,600 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:43,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:43,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:43,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:43,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,613 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:43,613 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:43,613 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:43,613 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:43,614 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,614 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,615 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,615 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:43,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:43,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:43,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:43,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:43,617 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:43,617 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:43,618 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:43,618 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:43,618 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:43,618 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:43,618 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:43,618 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:43,647 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:53:44,045 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 22:53:44,046 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 22:53:44,046 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201d532bc25a43f4dd78d9f4e0d2843f77e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201d532bc25a43f4dd78d9f4e0d2843f77e 2023-02-09 22:53:44,462 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/861b91faf5384f778791a94802bf8572 2023-02-09 22:53:44,471 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/861b91faf5384f778791a94802bf8572 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572 2023-02-09 22:53:44,479 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 22:53:44,480 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 862ms, sequenceid=13, compaction requested=true 2023-02-09 22:53:44,480 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:44,480 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:44,480 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:44,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:44,483 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:44,483 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,483 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:44,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:44,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@690a6a01[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:44,485 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:44,485 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,487 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:44,487 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:44,487 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:44,487 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,487 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:44,489 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:44,489 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,489 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:44,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:44,489 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:44,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:44,492 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:44,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:44,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:44,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,492 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:44,495 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,495 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,495 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:44,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:44,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:44,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:44,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,498 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:44,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:44,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:44,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:44,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:44,502 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:44,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:44,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:44,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:44,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:44,503 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:44,603 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:44,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:44,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:44,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:44,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,615 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:44,615 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:44,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:44,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:44,617 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,617 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,618 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:44,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:44,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:44,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,620 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:44,620 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:44,620 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:44,620 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:44,620 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:44,620 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:44,620 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:44,620 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:44,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:45,045 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205815e2c0c712d414cae475b7f449d1fa7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205815e2c0c712d414cae475b7f449d1fa7 2023-02-09 22:53:45,458 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/8054e32e87684cbb8607b1cab74f82ce 2023-02-09 22:53:45,467 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/8054e32e87684cbb8607b1cab74f82ce as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce 2023-02-09 22:53:45,476 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 22:53:45,477 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 856ms, sequenceid=17, compaction requested=true 2023-02-09 22:53:45,477 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:45,477 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:45,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:45,478 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:45,478 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,478 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:45,478 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:45,481 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:45,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:45,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:45,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:45,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:45,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:45,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:45,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67db6ba7[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:45,483 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:45,483 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,486 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,486 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:45,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:45,486 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:45,486 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:45,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:45,488 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:45,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:45,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:45,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:45,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:45,492 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:45,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:45,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:45,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,492 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:45,494 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,494 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:45,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,494 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:45,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:45,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,495 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,497 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:45,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:45,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:45,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,502 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:45,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:45,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:45,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:45,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:45,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:45,503 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:45,603 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:45,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:45,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:45,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:45,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,615 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:45,615 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:45,615 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:45,615 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:45,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,616 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,623 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,623 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:45,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:45,623 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:45,623 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:45,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:45,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,624 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,624 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:45,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:45,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:45,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:45,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:45,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:45,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:45,625 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:45,625 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:45,626 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:46,051 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122864512847c4c64cee971c8078a5f09a1a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122864512847c4c64cee971c8078a5f09a1a 2023-02-09 22:53:46,463 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/b68420f4878048e09c8496202b4aa23f 2023-02-09 22:53:46,472 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/b68420f4878048e09c8496202b4aa23f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f 2023-02-09 22:53:46,480 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 22:53:46,481 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 856ms, sequenceid=21, compaction requested=true 2023-02-09 22:53:46,481 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:46,481 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:46,481 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:46,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:46,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:46,482 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:46,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,487 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:46,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f554dac[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:46,490 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:46,490 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,491 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,491 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:46,491 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:46,491 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:46,491 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,491 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:46,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:46,494 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:46,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:46,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:46,498 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:46,498 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:46,498 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:46,498 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,498 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:46,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:46,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,500 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:46,500 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,502 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,504 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:46,504 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:46,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:46,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:46,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:46,504 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,505 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,505 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,505 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:46,506 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:46,506 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:46,606 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:46,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:46,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:46,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:46,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,618 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:46,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:46,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:46,618 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:46,619 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,619 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,620 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,620 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:46,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:46,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:46,621 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,621 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:46,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:46,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,622 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,622 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:46,622 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,622 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:46,622 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:46,622 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:46,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:46,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:46,623 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:46,623 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:46,623 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:46,644 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317dec58a6659945979ab7409af3b79340 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317dec58a6659945979ab7409af3b79340 2023-02-09 22:53:46,652 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/5d9a085aa4264687bae4c7ba18e8b1b9 2023-02-09 22:53:46,660 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/5d9a085aa4264687bae4c7ba18e8b1b9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9 2023-02-09 22:53:46,666 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 22:53:46,667 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 44ms, sequenceid=25, compaction requested=true 2023-02-09 22:53:46,668 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:46,668 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:46,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:46,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:46,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,669 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:46,669 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:46,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:46,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5733975[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:46,674 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:46,674 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,675 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,676 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:46,676 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:46,676 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:46,676 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,676 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:46,678 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:46,678 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,678 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,678 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:46,681 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:46,681 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:46,682 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:46,682 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:46,682 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,682 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:46,683 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,683 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:46,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,683 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:46,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:46,684 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:46,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,689 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:46,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:46,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:46,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:46,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:46,693 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:46,693 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:46,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,694 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:46,694 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:46,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:46,795 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:46,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:46,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:46,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:46,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,806 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:46,806 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:46,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:46,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:46,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,809 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,809 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:46,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:46,809 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:46,809 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,811 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:46,811 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:46,811 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:46,811 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:46,811 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:46,812 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:46,812 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:46,812 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:46,812 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:46,812 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:47,235 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018ed37f55cb1446d3b96576bf47b7a315 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018ed37f55cb1446d3b96576bf47b7a315 2023-02-09 22:53:47,645 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/e7f843d3c62d4378a2a21257ce6b3cd0 2023-02-09 22:53:47,654 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/e7f843d3c62d4378a2a21257ce6b3cd0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0 2023-02-09 22:53:47,662 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 22:53:47,692 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 880ms, sequenceid=29, compaction requested=true 2023-02-09 22:53:47,694 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:47,694 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:47,694 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:47,694 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:47,694 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,694 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:47,694 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:47,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,697 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:47,697 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,697 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:47,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:47,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f1311b8[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:47,699 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:47,699 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,700 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,700 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:47,700 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:47,700 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:47,700 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,700 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:47,703 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:47,703 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,703 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:47,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:47,703 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:47,706 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:47,706 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:47,706 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:47,706 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,707 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:47,708 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,708 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,708 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:47,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:47,709 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,710 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:47,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:47,715 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:47,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:47,715 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:47,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:47,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:47,716 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,716 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:47,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:47,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:47,717 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:47,717 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:47,817 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:47,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:47,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:47,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:47,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,828 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:47,828 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:47,828 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:47,828 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:47,829 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,829 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,835 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:47,835 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:47,835 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:47,836 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,836 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,837 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:47,837 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:47,837 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:47,837 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:47,837 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:47,837 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:47,837 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:47,837 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:47,838 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:47,838 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:48,259 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e8fba7aca0e94a27b3c29255744f10d8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103e8fba7aca0e94a27b3c29255744f10d8 2023-02-09 22:53:48,668 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/b2b746090db54ca0ba4b165787f88530 2023-02-09 22:53:48,676 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/b2b746090db54ca0ba4b165787f88530 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530 2023-02-09 22:53:48,682 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 22:53:48,684 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 846ms, sequenceid=33, compaction requested=true 2023-02-09 22:53:48,684 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:48,684 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:48,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:48,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:48,685 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,685 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:48,685 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:48,686 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,686 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:48,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:48,687 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,687 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:48,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:48,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:48,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:48,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:48,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@329d3ae[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:48,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:48,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,689 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:48,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:48,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:48,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:48,691 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:48,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,691 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:48,691 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:48,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:48,694 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:48,694 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:48,694 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:48,695 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,695 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:48,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:48,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:48,697 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,699 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:48,701 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:48,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:48,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:48,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:48,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,702 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:48,702 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:48,703 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:48,803 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:48,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:48,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:48,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:48,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,815 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:48,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:48,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:48,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:48,816 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,816 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,817 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:48,817 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:48,817 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:48,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:48,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,818 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:48,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:48,819 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:48,820 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:48,820 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:48,820 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:49,240 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079e94d97908aa4e81a14dd05f6a74cfeb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509079e94d97908aa4e81a14dd05f6a74cfeb 2023-02-09 22:53:49,649 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/9f6fe29df55e497ebf0d294bbff1b528 2023-02-09 22:53:49,658 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/9f6fe29df55e497ebf0d294bbff1b528 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528 2023-02-09 22:53:49,665 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 22:53:49,667 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 847ms, sequenceid=37, compaction requested=true 2023-02-09 22:53:49,667 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:49,667 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:49,667 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:49,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:49,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:49,668 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:49,669 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,670 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:49,670 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,670 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:49,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:49,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:49,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:49,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46d4c64a[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:49,671 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:49,671 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,673 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:49,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:49,673 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,673 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:49,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:49,675 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:49,675 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,675 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:49,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:49,675 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:49,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:49,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:49,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:49,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:49,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:49,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:49,678 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:49,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:49,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:49,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,678 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:49,686 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,686 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,686 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:49,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:49,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:49,687 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:49,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:49,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:49,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:49,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:49,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:49,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:49,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:49,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:49,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:49,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:49,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,691 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:49,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:49,692 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:49,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:49,792 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:49,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:49,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:49,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:49,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,804 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:49,804 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:49,804 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-09 22:53:49,804 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-09 22:53:49,805 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,805 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,807 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:49,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:49,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:49,807 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:49,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,808 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-09 22:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-09 22:53:49,809 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:49,810 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-09 22:53:49,810 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:49,810 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:49,810 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:49,810 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. started... 2023-02-09 22:53:49,810 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0912312fd88f712f69082600e46ca0c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:50,230 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a6303d7794d3454188264ed3b379a564 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6303d7794d3454188264ed3b379a564 2023-02-09 22:53:50,655 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/98a1370517a743e8823a21e5bba46f44 2023-02-09 22:53:50,663 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/98a1370517a743e8823a21e5bba46f44 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44 2023-02-09 22:53:50,669 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 22:53:50,670 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0912312fd88f712f69082600e46ca0c8 in 860ms, sequenceid=41, compaction requested=true 2023-02-09 22:53:50,671 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:50,671 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:50,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:50,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:50,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-09 22:53:50,671 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-09 22:53:50,673 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:50,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:50,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,673 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:50,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:50,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-09 22:53:50,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a70f774[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:50,675 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-09 22:53:50,675 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,677 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-09 22:53:50,677 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,677 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:50,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-09 22:53:50,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:50,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-09 22:53:50,682 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:50,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-09 22:53:50,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:50,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:50,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-09 22:53:50,686 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-09 22:53:50,687 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:50,687 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:50,687 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,687 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:50,688 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,688 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:50,688 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:50,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:50,689 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:50,694 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:50,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:50,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-09 22:53:50,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,694 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:50,695 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:50,695 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:50,795 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:50,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:50,801 DEBUG [Listener at localhost/33819] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:53:50,804 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41684, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:53:50,805 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:50,818 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:50,818 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:50,820 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:53:50,820 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983830820 2023-02-09 22:53:50,823 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:53:50,827 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 22:53:50,827 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:50,827 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x37261d86 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:50,831 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7605db8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:50,834 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,706 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ac6910cb36e54b479d0280140be64f1c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103ac6910cb36e54b479d0280140be64f1c 2023-02-09 22:53:51,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:51,726 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:53:51,728 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:35356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:53:51,739 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:53:51,741 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41694, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:53:51,776 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/647acba50c834feb82ff2d73f8bc846e first=Optional[r3] last=Optional[r6] 2023-02-09 22:53:51,789 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/647acba50c834feb82ff2d73f8bc846e}] 2023-02-09 22:53:51,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/647acba50c834feb82ff2d73f8bc846e for inclusion in 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:51,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 22:53:51,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:53:51,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:51,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/647acba50c834feb82ff2d73f8bc846e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__t8vr42r3j9bsp4ts14jei2b9ta0rddf7hf9jsg7j89l7clt2gvell3qoilhh148u/family1/647acba50c834feb82ff2d73f8bc846e 2023-02-09 22:53:51,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__t8vr42r3j9bsp4ts14jei2b9ta0rddf7hf9jsg7j89l7clt2gvell3qoilhh148u/family1/647acba50c834feb82ff2d73f8bc846e as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_ 2023-02-09 22:53:51,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__t8vr42r3j9bsp4ts14jei2b9ta0rddf7hf9jsg7j89l7clt2gvell3qoilhh148u/family1/647acba50c834feb82ff2d73f8bc846e into 0912312fd88f712f69082600e46ca0c8/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_ - updating store file list. 2023-02-09 22:53:51,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_ into 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:51,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__t8vr42r3j9bsp4ts14jei2b9ta0rddf7hf9jsg7j89l7clt2gvell3qoilhh148u/family1/647acba50c834feb82ff2d73f8bc846e into 0912312fd88f712f69082600e46ca0c8/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_) 2023-02-09 22:53:51,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__t8vr42r3j9bsp4ts14jei2b9ta0rddf7hf9jsg7j89l7clt2gvell3qoilhh148u/family1/647acba50c834feb82ff2d73f8bc846e 2023-02-09 22:53:51,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:53:51,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:53:51,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): 0912312fd88f712f69082600e46ca0c8/family1 is initiating minor compaction 2023-02-09 22:53:51,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7066f8e9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:53:51,836 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0912312fd88f712f69082600e46ca0c8/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:51,837 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp, totalSize=50.2 K 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 3920e07c91904bed9e88ca0e51bf8577, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 861b91faf5384f778791a94802bf8572, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 8054e32e87684cbb8607b1cab74f82ce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b68420f4878048e09c8496202b4aa23f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 5d9a085aa4264687bae4c7ba18e8b1b9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:53:51,838 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting e7f843d3c62d4378a2a21257ce6b3cd0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:53:51,839 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b2b746090db54ca0ba4b165787f88530, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:53:51,839 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 9f6fe29df55e497ebf0d294bbff1b528, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:53:51,839 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 98a1370517a743e8823a21e5bba46f44, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:53:51,839 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:53:51,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:51,852 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:53:51,859 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122864512847c4c64cee971c8078a5f09a1a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122864512847c4c64cee971c8078a5f09a1a 2023-02-09 22:53:51,861 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317dec58a6659945979ab7409af3b79340 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317dec58a6659945979ab7409af3b79340 2023-02-09 22:53:51,863 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018ed37f55cb1446d3b96576bf47b7a315 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018ed37f55cb1446d3b96576bf47b7a315 2023-02-09 22:53:51,864 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103e8fba7aca0e94a27b3c29255744f10d8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103e8fba7aca0e94a27b3c29255744f10d8 2023-02-09 22:53:51,864 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-09 22:53:51,864 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:51,888 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0912312fd88f712f69082600e46ca0c8#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:53:52,316 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/084790bd81554216a579ffc39cdbda26 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/084790bd81554216a579ffc39cdbda26 2023-02-09 22:53:52,330 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 0912312fd88f712f69082600e46ca0c8/family1 of 0912312fd88f712f69082600e46ca0c8 into 084790bd81554216a579ffc39cdbda26(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:53:52,330 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:52,332 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1675983231834; duration=0sec 2023-02-09 22:53:52,333 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:53:52,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_] to archive 2023-02-09 22:53:52,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:53:52,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/3920e07c91904bed9e88ca0e51bf8577 2023-02-09 22:53:52,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/861b91faf5384f778791a94802bf8572 2023-02-09 22:53:52,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/8054e32e87684cbb8607b1cab74f82ce 2023-02-09 22:53:52,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b68420f4878048e09c8496202b4aa23f 2023-02-09 22:53:52,393 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/5d9a085aa4264687bae4c7ba18e8b1b9 2023-02-09 22:53:52,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/e7f843d3c62d4378a2a21257ce6b3cd0 2023-02-09 22:53:52,395 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/b2b746090db54ca0ba4b165787f88530 2023-02-09 22:53:52,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/9f6fe29df55e497ebf0d294bbff1b528 2023-02-09 22:53:52,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/98a1370517a743e8823a21e5bba46f44 2023-02-09 22:53:52,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/0f47b0724f32407ba8a95a8d2dcce53c_SeqId_44_ 2023-02-09 22:53:52,750 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059308dfd30d564166a2121a1c2d90dc7a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512059308dfd30d564166a2121a1c2d90dc7a 2023-02-09 22:53:52,752 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:52,781 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/11474641f8354161a2031ce99f7b990d first=Optional[r00] last=Optional[r2] 2023-02-09 22:53:52,782 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/11474641f8354161a2031ce99f7b990d}] 2023-02-09 22:53:52,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/11474641f8354161a2031ce99f7b990d for inclusion in 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:52,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 22:53:52,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:53:52,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:52,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/11474641f8354161a2031ce99f7b990d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__a6hnn72dhhag8jojvlffrg19g9qk3jbo41o2e9e1vse72e0ddu78pb4lac01mtmk/family1/11474641f8354161a2031ce99f7b990d 2023-02-09 22:53:52,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__a6hnn72dhhag8jojvlffrg19g9qk3jbo41o2e9e1vse72e0ddu78pb4lac01mtmk/family1/11474641f8354161a2031ce99f7b990d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_ 2023-02-09 22:53:52,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__a6hnn72dhhag8jojvlffrg19g9qk3jbo41o2e9e1vse72e0ddu78pb4lac01mtmk/family1/11474641f8354161a2031ce99f7b990d into 0912312fd88f712f69082600e46ca0c8/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_ - updating store file list. 2023-02-09 22:53:52,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_ into 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:52,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__a6hnn72dhhag8jojvlffrg19g9qk3jbo41o2e9e1vse72e0ddu78pb4lac01mtmk/family1/11474641f8354161a2031ce99f7b990d into 0912312fd88f712f69082600e46ca0c8/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_) 2023-02-09 22:53:52,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__a6hnn72dhhag8jojvlffrg19g9qk3jbo41o2e9e1vse72e0ddu78pb4lac01mtmk/family1/11474641f8354161a2031ce99f7b990d 2023-02-09 22:53:52,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:53:52,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:53:52,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): 0912312fd88f712f69082600e46ca0c8/family1 is initiating minor compaction (all files) 2023-02-09 22:53:52,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@ffc77fe; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:53:52,805 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0912312fd88f712f69082600e46ca0c8/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:52,806 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/084790bd81554216a579ffc39cdbda26, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp, totalSize=358.6 K 2023-02-09 22:53:52,807 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 05bf8894f1be4ceb99ecfcbc9ed96c59, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:53:52,807 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 084790bd81554216a579ffc39cdbda26, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:53:52,808 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 01a2db0008634c498115b9d0951d731d_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839414 2023-02-09 22:53:52,808 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:52,812 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:53:52,814 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec5257478a334b5fb47701000aa71b75 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130ec5257478a334b5fb47701000aa71b75 2023-02-09 22:53:52,817 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201d532bc25a43f4dd78d9f4e0d2843f77e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201d532bc25a43f4dd78d9f4e0d2843f77e 2023-02-09 22:53:52,818 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205815e2c0c712d414cae475b7f449d1fa7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205815e2c0c712d414cae475b7f449d1fa7 2023-02-09 22:53:52,818 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-09 22:53:52,818 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 22:53:52,819 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:52,819 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:52,819 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:53:52,820 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:52,823 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0110 sec 2023-02-09 22:53:52,846 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0912312fd88f712f69082600e46ca0c8#family1#compaction#11 average throughput is 7.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:53:52,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 22:53:52,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:53,394 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 22:53:53,395 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:53,395 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-09 22:53:53,400 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-09 22:53:53,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, REOPEN/MOVE}] 2023-02-09 22:53:53,413 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, REOPEN/MOVE 2023-02-09 22:53:53,414 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:53,414 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983233414"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983233414"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983233414"}]},"ts":"1675983233414"} 2023-02-09 22:53:53,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:53,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:53,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 0912312fd88f712f69082600e46ca0c8, disabling compactions & flushes 2023-02-09 22:53:53,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,080 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/.tmp/family1/eaeab4c2b8cd414fb7596425a5767545 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/eaeab4c2b8cd414fb7596425a5767545 2023-02-09 22:53:54,087 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 0912312fd88f712f69082600e46ca0c8/family1 of 0912312fd88f712f69082600e46ca0c8 into eaeab4c2b8cd414fb7596425a5767545(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:53:54,088 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:54,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. after waiting 0 ms 2023-02-09 22:53:54,088 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1675983232805; duration=1sec 2023-02-09 22:53:54,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,088 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:53:54,090 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/084790bd81554216a579ffc39cdbda26, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_] to archive 2023-02-09 22:53:54,091 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:53:54,093 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/05bf8894f1be4ceb99ecfcbc9ed96c59 2023-02-09 22:53:54,095 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/084790bd81554216a579ffc39cdbda26 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/084790bd81554216a579ffc39cdbda26 2023-02-09 22:53:54,097 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/01a2db0008634c498115b9d0951d731d_SeqId_47_ 2023-02-09 22:53:54,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 22:53:54,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 0912312fd88f712f69082600e46ca0c8: Waiting for close lock at 1675983233575Running coprocessor pre-close hooks at 1675983233575Disabling compacts and flushes for region at 1675983233575Disabling writes for close at 1675983234088 (+513 ms)Writing region close event to WAL at 1675983234100 (+12 ms)Running coprocessor post-close hooks at 1675983234104 (+4 ms)Closed at 1675983234105 (+1 ms) 2023-02-09 22:53:54,105 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 0912312fd88f712f69082600e46ca0c8 to self. 2023-02-09 22:53:54,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,108 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=CLOSED 2023-02-09 22:53:54,108 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983234108"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983234108"}]},"ts":"1675983234108"} 2023-02-09 22:53:54,112 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-09 22:53:54,112 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026 in 692 msec 2023-02-09 22:53:54,113 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=true 2023-02-09 22:53:54,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:54,263 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983234263"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983234263"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983234263"}]},"ts":"1675983234263"} 2023-02-09 22:53:54,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:54,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0912312fd88f712f69082600e46ca0c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,424 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,425 DEBUG [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:54,425 DEBUG [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:54,426 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0912312fd88f712f69082600e46ca0c8 columnFamilyName family1 2023-02-09 22:53:54,432 DEBUG [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/eaeab4c2b8cd414fb7596425a5767545 2023-02-09 22:53:54,433 INFO [StoreOpener-0912312fd88f712f69082600e46ca0c8-1] regionserver.HStore(310): Store=0912312fd88f712f69082600e46ca0c8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:54,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:53:54,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 0912312fd88f712f69082600e46ca0c8; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69678834, jitterRate=0.03829553723335266}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:53:54,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 0912312fd88f712f69082600e46ca0c8: Running coprocessor pre-open hook at 1675983234423Writing region info on filesystem at 1675983234423Initializing all the Stores at 1675983234424 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675983234424Cleaning up temporary data from old regions at 1675983234436 (+12 ms)Cleaning up detritus from prior splits at 1675983234436Running coprocessor post-open hooks at 1675983234439 (+3 ms)Region opened successfully at 1675983234439 2023-02-09 22:53:54,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., pid=17, masterSystemTime=1675983234418 2023-02-09 22:53:54,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:53:54,442 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0912312fd88f712f69082600e46ca0c8, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:54,442 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675983234442"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983234442"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983234442"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983234442"}]},"ts":"1675983234442"} 2023-02-09 22:53:54,446 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-09 22:53:54,446 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 0912312fd88f712f69082600e46ca0c8, server=jenkins-hbase9.apache.org,39093,1675983215026 in 178 msec 2023-02-09 22:53:54,448 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-09 22:53:54,448 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0912312fd88f712f69082600e46ca0c8, REOPEN/MOVE in 1.0340 sec 2023-02-09 22:53:54,451 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 22:53:54,451 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0500 sec 2023-02-09 22:53:54,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4850 sec 2023-02-09 22:53:54,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-09 22:53:54,463 INFO [Listener at localhost/33819] compactions.TestMobCompactor(278): alter status finished 2023-02-09 22:53:54,464 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-09 22:53:54,465 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:54,466 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:54,467 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:53:54,467 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983834467 2023-02-09 22:53:54,468 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:53:54,469 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-09 22:53:54,469 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:54,470 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x688a6e25 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:53:54,475 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@697a5079, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:53:54,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,475 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,912 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a8a1ffa4eb6549c4b733dc2ffb7fecdc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128a8a1ffa4eb6549c4b733dc2ffb7fecdc 2023-02-09 22:53:54,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:54,927 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:53:54,928 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:35358, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:53:54,936 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:53:54,937 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41702, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:53:54,948 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1ad017ebf6c14be7bb1c4434cbc37fcc first=Optional[r05] last=Optional[r8] 2023-02-09 22:53:54,948 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1ad017ebf6c14be7bb1c4434cbc37fcc}] 2023-02-09 22:53:54,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1ad017ebf6c14be7bb1c4434cbc37fcc for inclusion in 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:54,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 22:53:54,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:53:54,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 0912312fd88f712f69082600e46ca0c8: 2023-02-09 22:53:54,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1ad017ebf6c14be7bb1c4434cbc37fcc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f3u5kj2n8meuvpl35bg8guanabuedbljl72st2db3mtnjim0s82ioo3ma9b7im7l/family1/1ad017ebf6c14be7bb1c4434cbc37fcc 2023-02-09 22:53:54,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f3u5kj2n8meuvpl35bg8guanabuedbljl72st2db3mtnjim0s82ioo3ma9b7im7l/family1/1ad017ebf6c14be7bb1c4434cbc37fcc as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/4bac2bc51fb04fe1a5d0f4686febe14e_SeqId_53_ 2023-02-09 22:53:54,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f3u5kj2n8meuvpl35bg8guanabuedbljl72st2db3mtnjim0s82ioo3ma9b7im7l/family1/1ad017ebf6c14be7bb1c4434cbc37fcc into 0912312fd88f712f69082600e46ca0c8/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/4bac2bc51fb04fe1a5d0f4686febe14e_SeqId_53_ - updating store file list. 2023-02-09 22:53:54,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/4bac2bc51fb04fe1a5d0f4686febe14e_SeqId_53_ into 0912312fd88f712f69082600e46ca0c8/family1 2023-02-09 22:53:54,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f3u5kj2n8meuvpl35bg8guanabuedbljl72st2db3mtnjim0s82ioo3ma9b7im7l/family1/1ad017ebf6c14be7bb1c4434cbc37fcc into 0912312fd88f712f69082600e46ca0c8/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/family1/4bac2bc51fb04fe1a5d0f4686febe14e_SeqId_53_) 2023-02-09 22:53:54,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__f3u5kj2n8meuvpl35bg8guanabuedbljl72st2db3mtnjim0s82ioo3ma9b7im7l/family1/1ad017ebf6c14be7bb1c4434cbc37fcc 2023-02-09 22:53:54,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:54,971 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:53:54,973 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6303d7794d3454188264ed3b379a564 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120a6303d7794d3454188264ed3b379a564 2023-02-09 22:53:54,974 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128888e1a3a132b492a9e291a8bc80c53f3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128888e1a3a132b492a9e291a8bc80c53f3 2023-02-09 22:53:54,974 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-09 22:53:54,975 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 22:53:54,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-09 22:53:54,975 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-09 22:53:54,975 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:53:54,976 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-09 22:53:54,978 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 512 msec 2023-02-09 22:53:55,060 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 211) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x37261d86-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:48052 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47968 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47992 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:48008 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7f4ff478-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x688a6e25 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1888572195.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:48090 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x688a6e25-SendThread(127.0.0.1:59035) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4995f1-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-209665605_15 at /127.0.0.1:47876 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x37261d86-SendThread(127.0.0.1:59035) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS:0;jenkins-hbase9:39093-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47904 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:35140 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4995f1-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47954 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47990 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47982 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7f4ff478-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4995f1-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:35118 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:48022 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x688a6e25-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x37261d86 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1888572195.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47938 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:48098 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=585 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=240 (was 335), ProcessCount=168 (was 168), AvailableMemoryMB=11651 (was 11776) 2023-02-09 22:53:55,072 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=585, MaxFileDescriptor=60000, SystemLoadAverage=240, ProcessCount=168, AvailableMemoryMB=11651 2023-02-09 22:53:55,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 22:53:55,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:55,077 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:53:55,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-09 22:53:55,078 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:53:55,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:55,080 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:55,081 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e empty. 2023-02-09 22:53:55,082 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:55,082 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-09 22:53:55,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:55,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:55,492 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 22:53:55,494 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => de6f52acd092b2a4a3ed68bb356a064e, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:53:55,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:55,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:55,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing de6f52acd092b2a4a3ed68bb356a064e, disabling compactions & flushes 2023-02-09 22:53:55,906 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:55,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:55,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. after waiting 0 ms 2023-02-09 22:53:55,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:55,906 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:55,906 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for de6f52acd092b2a4a3ed68bb356a064e: Waiting for close lock at 1675983235905Disabling compacts and flushes for region at 1675983235905Disabling writes for close at 1675983235906 (+1 ms)Writing region close event to WAL at 1675983235906Closed at 1675983235906 2023-02-09 22:53:55,909 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:53:55,910 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983235909"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983235909"}]},"ts":"1675983235909"} 2023-02-09 22:53:55,912 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:53:55,913 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:53:55,913 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983235913"}]},"ts":"1675983235913"} 2023-02-09 22:53:55,914 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 22:53:55,917 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=de6f52acd092b2a4a3ed68bb356a064e, ASSIGN}] 2023-02-09 22:53:55,918 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=de6f52acd092b2a4a3ed68bb356a064e, ASSIGN 2023-02-09 22:53:55,919 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=de6f52acd092b2a4a3ed68bb356a064e, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:53:56,070 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=de6f52acd092b2a4a3ed68bb356a064e, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:56,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983236070"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983236070"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983236070"}]},"ts":"1675983236070"} 2023-02-09 22:53:56,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure de6f52acd092b2a4a3ed68bb356a064e, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:53:56,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:56,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:56,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => de6f52acd092b2a4a3ed68bb356a064e, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:53:56,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:53:56,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,231 INFO [StoreOpener-de6f52acd092b2a4a3ed68bb356a064e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,232 DEBUG [StoreOpener-de6f52acd092b2a4a3ed68bb356a064e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:53:56,233 DEBUG [StoreOpener-de6f52acd092b2a4a3ed68bb356a064e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:53:56,233 INFO [StoreOpener-de6f52acd092b2a4a3ed68bb356a064e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region de6f52acd092b2a4a3ed68bb356a064e columnFamilyName family1 2023-02-09 22:53:56,235 INFO [StoreOpener-de6f52acd092b2a4a3ed68bb356a064e-1] regionserver.HStore(310): Store=de6f52acd092b2a4a3ed68bb356a064e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:53:56,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:53:56,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:53:56,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened de6f52acd092b2a4a3ed68bb356a064e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70436670, jitterRate=0.049588173627853394}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:53:56,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for de6f52acd092b2a4a3ed68bb356a064e: Running coprocessor pre-open hook at 1675983236230Writing region info on filesystem at 1675983236230Initializing all the Stores at 1675983236230Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675983236231 (+1 ms)Cleaning up temporary data from old regions at 1675983236236 (+5 ms)Cleaning up detritus from prior splits at 1675983236237 (+1 ms)Running coprocessor post-open hooks at 1675983236241 (+4 ms)Region opened successfully at 1675983236241 2023-02-09 22:53:56,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., pid=21, masterSystemTime=1675983236225 2023-02-09 22:53:56,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:56,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:56,245 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=de6f52acd092b2a4a3ed68bb356a064e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:56,245 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983236245"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983236245"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983236245"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983236245"}]},"ts":"1675983236245"} 2023-02-09 22:53:56,249 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-09 22:53:56,249 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure de6f52acd092b2a4a3ed68bb356a064e, server=jenkins-hbase9.apache.org,39093,1675983215026 in 174 msec 2023-02-09 22:53:56,251 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-09 22:53:56,251 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=de6f52acd092b2a4a3ed68bb356a064e, ASSIGN in 332 msec 2023-02-09 22:53:56,252 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:53:56,252 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983236252"}]},"ts":"1675983236252"} 2023-02-09 22:53:56,253 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 22:53:56,256 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:53:56,257 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1810 sec 2023-02-09 22:53:57,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 22:53:57,186 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-09 22:53:57,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:57,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,226 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,226 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,227 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:57,227 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:57,228 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,228 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,229 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:57,229 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:57,229 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,229 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,231 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:57,231 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,231 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:57,232 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:57,232 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:57,232 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:57,232 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:57,232 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:53:57,666 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fdbf772afacc45d49d594788e8a2e594 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130fdbf772afacc45d49d594788e8a2e594 2023-02-09 22:53:57,670 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a8c55c0456ce4cf7865377ae59d0b73a 2023-02-09 22:53:57,677 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a8c55c0456ce4cf7865377ae59d0b73a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a 2023-02-09 22:53:57,682 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 22:53:57,683 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 451ms, sequenceid=5, compaction requested=false 2023-02-09 22:53:57,683 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,684 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:53:57,684 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:57,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:57,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:57,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:53:57,684 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:53:57,686 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,686 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,686 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,687 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dc9a353[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:57,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:53:57,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,689 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:53:57,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,690 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:57,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:53:57,690 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:57,692 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,692 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,692 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:57,692 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:53:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:57,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:53:57,695 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:53:57,695 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:57,695 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:57,695 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,695 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:57,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,697 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,698 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,699 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:57,701 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:57,702 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:57,702 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:57,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:57,702 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,702 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:57,702 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:57,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:53:57,703 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,703 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:57,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:57,703 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:57,803 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:57,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:57,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:57,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:57,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,812 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,812 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,813 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:57,813 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:57,813 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,813 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,815 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:57,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:57,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,815 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:57,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,816 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:57,816 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,817 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:57,817 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:57,817 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:57,817 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:57,817 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:57,817 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:53:57,835 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283b02b43ae7914fa6a0aa6fef22b9878d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283b02b43ae7914fa6a0aa6fef22b9878d 2023-02-09 22:53:57,842 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/03d845b329c64f63bb30434c67b2eedd 2023-02-09 22:53:57,848 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/03d845b329c64f63bb30434c67b2eedd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd 2023-02-09 22:53:57,854 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:53:57,855 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 38ms, sequenceid=9, compaction requested=false 2023-02-09 22:53:57,855 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:53:57,855 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:57,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:57,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:57,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:53:57,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:53:57,857 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,858 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,858 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,858 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:53:57,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3030abe7[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:57,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:53:57,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,861 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,861 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:53:57,861 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:53:57,861 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,861 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:57,861 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:57,862 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:53:57,862 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,862 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,863 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:57,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:53:57,866 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:53:57,866 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:57,866 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:57,866 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,866 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:57,867 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,867 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,867 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:57,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:57,868 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,869 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:57,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:57,877 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:57,877 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:57,878 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:57,878 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:57,878 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:57,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:57,878 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,878 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,978 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:57,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:57,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:57,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:57,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,987 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:57,987 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,988 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:57,988 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:57,988 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,988 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,989 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:57,990 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:57,990 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:57,990 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:53:57,992 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:57,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:57,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:57,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:57,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:57,993 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:57,993 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:57,993 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:57,993 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:58,014 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120133b4717be73049878208d7bcd5bc6aa2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120133b4717be73049878208d7bcd5bc6aa2 2023-02-09 22:53:58,021 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/fd434ebde62145bf8b82ca3811e7cb49 2023-02-09 22:53:58,028 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/fd434ebde62145bf8b82ca3811e7cb49 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49 2023-02-09 22:53:58,038 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 22:53:58,039 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 46ms, sequenceid=13, compaction requested=true 2023-02-09 22:53:58,039 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:53:58,039 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:58,039 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:58,039 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:58,039 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,039 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:53:58,039 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:53:58,041 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:58,042 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,042 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:58,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:53:58,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50b6cd3d[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:58,043 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:53:58,043 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,045 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,045 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:53:58,045 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:53:58,045 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:58,047 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:53:58,047 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:58,047 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:53:58,051 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:53:58,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:58,052 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:53:58,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:58,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:58,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,052 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:58,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,054 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:58,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,054 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:58,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:58,054 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,056 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,058 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:58,058 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:58,059 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:58,058 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:58,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:58,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:58,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:58,059 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:58,060 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:58,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:58,160 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:58,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:58,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:58,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:58,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,176 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:58,176 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:58,177 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:58,177 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:58,178 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,178 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,179 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,179 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:58,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:58,179 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,179 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:58,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,180 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:58,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:53:58,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:58,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:58,188 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:58,188 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:58,189 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:58,189 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:58,189 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:58,189 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:58,209 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205153d5d7452b742bf81c1f2e90081aafe to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205153d5d7452b742bf81c1f2e90081aafe 2023-02-09 22:53:58,620 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a95d5f3f64a7414a8e608cde37e806e0 2023-02-09 22:53:58,626 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a95d5f3f64a7414a8e608cde37e806e0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0 2023-02-09 22:53:58,632 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 22:53:58,634 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 445ms, sequenceid=17, compaction requested=true 2023-02-09 22:53:58,634 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:53:58,634 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:58,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:58,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:58,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:53:58,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:53:58,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,641 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:58,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:58,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,642 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:58,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:58,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:58,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:58,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:53:58,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d204b08[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:58,643 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:53:58,643 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:53:58,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:58,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:53:58,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:58,646 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,646 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:53:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:58,647 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:58,647 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:53:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:58,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:58,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:58,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:53:58,651 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:53:58,651 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:58,651 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:58,651 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,651 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:58,652 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,652 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:58,653 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:58,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,653 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,655 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,663 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:58,663 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:58,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:58,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:58,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:58,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,667 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,667 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:58,669 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:58,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:53:58,769 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:58,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:58,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:58,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:58,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,779 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:58,779 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:58,780 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:58,780 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:58,780 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,780 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,782 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:58,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:58,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:58,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:58,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:58,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,783 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,784 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:53:58,784 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:58,784 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:58,784 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:58,785 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:58,788 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:58,788 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:58,788 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:58,788 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:58,788 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:53:59,215 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512282dc9cf52770e45558343ad2ce9f11a8a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512282dc9cf52770e45558343ad2ce9f11a8a 2023-02-09 22:53:59,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/0b61b2ce2803425183d1adb6668246e9 2023-02-09 22:53:59,628 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/0b61b2ce2803425183d1adb6668246e9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9 2023-02-09 22:53:59,634 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 22:53:59,635 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 847ms, sequenceid=21, compaction requested=true 2023-02-09 22:53:59,635 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:53:59,635 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:59,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:53:59,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:53:59,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:53:59,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:53:59,638 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:59,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:53:59,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:59,640 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:53:59,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d43b2c4[Count = 0] remaining members to acquire global barrier 2023-02-09 22:53:59,641 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:53:59,641 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,647 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:53:59,648 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,648 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:53:59,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:53:59,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:53:59,650 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:53:59,650 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:59,650 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:53:59,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:59,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:59,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:53:59,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:53:59,654 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:53:59,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:53:59,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:53:59,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,654 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:53:59,666 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,666 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,666 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:53:59,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,667 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:59,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:53:59,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:53:59,669 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:53:59,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:59,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:59,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:53:59,671 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:53:59,671 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,672 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:53:59,673 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:53:59,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:53:59,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:53:59,673 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,673 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:53:59,673 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,773 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:53:59,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:53:59,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:53:59,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:53:59,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,781 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:53:59,781 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:59,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:53:59,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:53:59,782 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,783 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,784 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:53:59,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:53:59,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:53:59,785 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:53:59,785 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:53:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:53:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,785 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,786 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:53:59,786 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:53:59,786 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:53:59,786 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:53:59,786 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:53:59,786 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:53:59,786 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:53:59,787 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:53:59,787 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:53:59,787 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:00,203 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317a2142620de741ed9ccedab40b11faf3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317a2142620de741ed9ccedab40b11faf3 2023-02-09 22:54:00,609 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/70f4c8c0df5d468f8551f4a54b07fe79 2023-02-09 22:54:00,616 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/70f4c8c0df5d468f8551f4a54b07fe79 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79 2023-02-09 22:54:00,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 22:54:00,623 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 836ms, sequenceid=25, compaction requested=true 2023-02-09 22:54:00,623 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:00,623 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:00,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:00,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:00,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:54:00,623 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:54:00,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,625 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,625 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:00,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:54:00,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8194918[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:00,627 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:00,627 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,634 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:00,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:54:00,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:00,634 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,634 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:00,636 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:54:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:54:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:00,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:00,638 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:54:00,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:00,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:00,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,639 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:00,640 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,640 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,640 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:00,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:00,641 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,644 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,644 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:00,644 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:00,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:00,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:00,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:00,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:00,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:00,646 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,646 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,646 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:00,646 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,646 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:00,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:00,746 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:00,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:00,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:00,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:00,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,753 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:00,754 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:00,754 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:54:00,754 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:00,755 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,755 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,757 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:00,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:00,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:00,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:00,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,758 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:54:00,758 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,758 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:54:00,758 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:00,759 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:54:00,759 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:00,759 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:00,759 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:00,759 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:54:00,760 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:00,790 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ba972f52dc0f42e5bc25c17f1e3b5093 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba972f52dc0f42e5bc25c17f1e3b5093 2023-02-09 22:54:00,811 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/57b08953287d4538b82e8dc2e61d17bf 2023-02-09 22:54:00,817 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/57b08953287d4538b82e8dc2e61d17bf as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf 2023-02-09 22:54:00,823 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 22:54:00,824 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 64ms, sequenceid=29, compaction requested=true 2023-02-09 22:54:00,824 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:00,824 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:00,824 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:00,824 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:00,824 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,824 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:54:00,824 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:54:00,826 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,826 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,826 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,826 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:54:00,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ea0b203[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:00,828 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:00,828 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,829 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,829 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:00,829 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:54:00,829 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:00,829 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,830 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:00,831 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,831 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:54:00,831 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,831 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:54:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:00,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:00,834 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:54:00,834 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:00,834 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:00,834 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,834 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:00,836 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,836 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:00,836 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:00,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:00,837 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:00,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,838 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,841 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,841 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,841 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:00,841 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:00,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:00,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:00,841 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:54:00,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:00,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:00,842 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:00,841 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:00,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:00,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:00,942 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:00,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:00,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:00,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:00,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,951 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:00,951 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:00,951 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:54:00,952 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:00,952 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,952 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,953 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:00,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:00,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:00,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:00,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,955 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:54:00,955 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:00,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:54:00,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:00,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:54:00,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:00,957 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:00,957 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:00,957 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:54:00,957 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:01,373 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103662e07fab98b4135a116e22e6cb45524 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103662e07fab98b4135a116e22e6cb45524 2023-02-09 22:54:01,780 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/2393b0a2a8e54e35b839f758e647d8f0 2023-02-09 22:54:01,786 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/2393b0a2a8e54e35b839f758e647d8f0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0 2023-02-09 22:54:01,791 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 22:54:01,792 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 835ms, sequenceid=33, compaction requested=true 2023-02-09 22:54:01,793 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:01,793 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:01,793 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:01,793 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:01,793 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,793 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:54:01,793 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:54:01,794 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,794 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:01,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:01,795 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,795 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:01,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:01,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:01,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:01,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:54:01,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6157ccd4[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:01,796 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:01,796 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,798 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,799 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:01,799 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:54:01,799 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,799 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:01,799 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:01,802 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:54:01,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:01,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:01,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:01,803 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:54:01,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:01,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:01,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:01,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:01,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:01,806 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:54:01,806 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:01,806 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:01,806 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,806 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:01,808 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,808 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:01,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:01,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,808 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:01,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:01,809 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:01,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:01,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:01,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,810 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:01,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:01,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:01,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:01,813 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,813 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:01,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:54:01,814 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:01,814 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,814 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:01,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:01,814 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,814 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:01,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:01,914 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:01,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:01,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:01,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:01,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,922 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:01,922 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:01,923 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:54:01,923 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:01,923 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,923 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,925 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,925 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:01,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:01,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:01,925 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:01,925 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:01,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,926 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:01,926 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:54:01,926 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:01,927 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:54:01,927 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:01,927 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:01,927 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:01,927 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:54:01,927 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:01,942 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907acacddb4b72146b3906b789e4a665ede to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907acacddb4b72146b3906b789e4a665ede 2023-02-09 22:54:02,356 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a7bd3e7fca764acebab36fdf7d0e2b53 2023-02-09 22:54:02,363 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/a7bd3e7fca764acebab36fdf7d0e2b53 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53 2023-02-09 22:54:02,370 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 22:54:02,371 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 444ms, sequenceid=37, compaction requested=true 2023-02-09 22:54:02,371 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:02,371 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:02,371 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:02,371 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:02,371 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,371 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:54:02,371 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:54:02,373 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,373 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:02,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:02,373 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,373 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:02,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:02,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:02,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:02,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:54:02,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41a297a8[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:02,375 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:02,375 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,376 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:02,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:54:02,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:02,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:02,379 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:54:02,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:02,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:02,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:02,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:54:02,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:02,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:02,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:02,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:02,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:02,383 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:54:02,383 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:02,383 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:02,383 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,383 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:02,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:02,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:02,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:02,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:02,385 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:02,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,387 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:02,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,390 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:02,390 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:02,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:02,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:02,390 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:02,391 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,391 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,391 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,391 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:02,391 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:02,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:02,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:02,491 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:02,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:02,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:02,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:02,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,504 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:02,504 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:02,506 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:54:02,506 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:02,506 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,506 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,508 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,508 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:02,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:02,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:02,508 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:02,508 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:02,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:02,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:54:02,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:02,510 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:54:02,510 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:02,511 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:54:02,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:02,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:02,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:02,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. started... 2023-02-09 22:54:02,515 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing de6f52acd092b2a4a3ed68bb356a064e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:02,940 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b0c8bdf674274bab9177459421955b84 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120b0c8bdf674274bab9177459421955b84 2023-02-09 22:54:03,347 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/c59ae7da41f74bd7a0d37b4dd9ae9148 2023-02-09 22:54:03,354 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/c59ae7da41f74bd7a0d37b4dd9ae9148 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148 2023-02-09 22:54:03,361 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 22:54:03,362 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for de6f52acd092b2a4a3ed68bb356a064e in 847ms, sequenceid=41, compaction requested=true 2023-02-09 22:54:03,362 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:03,362 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:03,362 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:03,362 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:03,362 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,362 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:54:03,362 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:54:03,365 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,365 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:03,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:03,366 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,366 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:03,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:03,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:54:03,367 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:03,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@331570d[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:03,367 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,368 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:03,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:54:03,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:03,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:03,370 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,370 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:54:03,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,370 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:03,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:03,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:03,370 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:54:03,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:03,373 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:03,373 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:54:03,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:03,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:03,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,374 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:03,375 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,375 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:03,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:03,375 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:03,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:03,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,377 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:03,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:03,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:03,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:03,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:54:03,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-09 22:54:03,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:03,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:03,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:03,382 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:03,483 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:03,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:03,485 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-09 22:54:03,487 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:03,487 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:03,488 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:03,488 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983843488 2023-02-09 22:54:03,490 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:03,492 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 22:54:03,492 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:03,492 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x66e9e9e9 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:03,497 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5dab94d5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:03,497 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:04,341 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103aab936c2900e4abaae34dd7cb51a7811 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103aab936c2900e4abaae34dd7cb51a7811 2023-02-09 22:54:04,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:04,359 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:04,360 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:37086, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:04,368 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:04,369 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33116, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:04,381 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 first=Optional[r5] last=Optional[r6] 2023-02-09 22:54:04,382 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c2d6fe95e7924fbc9f452949cd6a8ce2}] 2023-02-09 22:54:04,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 for inclusion in de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:04,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 22:54:04,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:04,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:04,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__g2ile9fucmerfgfmkag7rjae0htnu19835stj97vg779ieb55kam8odotntr7r88/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 2023-02-09 22:54:04,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__g2ile9fucmerfgfmkag7rjae0htnu19835stj97vg779ieb55kam8odotntr7r88/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_ 2023-02-09 22:54:04,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__g2ile9fucmerfgfmkag7rjae0htnu19835stj97vg779ieb55kam8odotntr7r88/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 into de6f52acd092b2a4a3ed68bb356a064e/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_ - updating store file list. 2023-02-09 22:54:04,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_ into de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:04,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__g2ile9fucmerfgfmkag7rjae0htnu19835stj97vg779ieb55kam8odotntr7r88/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 into de6f52acd092b2a4a3ed68bb356a064e/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_) 2023-02-09 22:54:04,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__g2ile9fucmerfgfmkag7rjae0htnu19835stj97vg779ieb55kam8odotntr7r88/family1/c2d6fe95e7924fbc9f452949cd6a8ce2 2023-02-09 22:54:04,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:54:04,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:54:04,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): de6f52acd092b2a4a3ed68bb356a064e/family1 is initiating minor compaction 2023-02-09 22:54:04,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@60ed9a2a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:04,405 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of de6f52acd092b2a4a3ed68bb356a064e/family1 in testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:04,406 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp, totalSize=49.7 K 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 03d845b329c64f63bb30434c67b2eedd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting fd434ebde62145bf8b82ca3811e7cb49, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting a95d5f3f64a7414a8e608cde37e806e0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 0b61b2ce2803425183d1adb6668246e9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 70f4c8c0df5d468f8551f4a54b07fe79, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 57b08953287d4538b82e8dc2e61d17bf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2393b0a2a8e54e35b839f758e647d8f0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting a7bd3e7fca764acebab36fdf7d0e2b53, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:54:04,406 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c59ae7da41f74bd7a0d37b4dd9ae9148, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:54:04,407 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b627ca6096764a69808e95b105dc5cda_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:54:04,408 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:04,409 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:04,412 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba972f52dc0f42e5bc25c17f1e3b5093 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba972f52dc0f42e5bc25c17f1e3b5093 2023-02-09 22:54:04,413 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103662e07fab98b4135a116e22e6cb45524 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103662e07fab98b4135a116e22e6cb45524 2023-02-09 22:54:04,413 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 22:54:04,413 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 22:54:04,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:04,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:04,462 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): de6f52acd092b2a4a3ed68bb356a064e#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:04,477 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/16c9970bd6a3479285ea607798804d8a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/16c9970bd6a3479285ea607798804d8a 2023-02-09 22:54:04,483 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in de6f52acd092b2a4a3ed68bb356a064e/family1 of de6f52acd092b2a4a3ed68bb356a064e into 16c9970bd6a3479285ea607798804d8a(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:04,483 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:04,484 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675983244405; duration=0sec 2023-02-09 22:54:04,484 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:04,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_] to archive 2023-02-09 22:54:04,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:04,590 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/03d845b329c64f63bb30434c67b2eedd 2023-02-09 22:54:04,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/fd434ebde62145bf8b82ca3811e7cb49 2023-02-09 22:54:04,592 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a95d5f3f64a7414a8e608cde37e806e0 2023-02-09 22:54:04,594 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/0b61b2ce2803425183d1adb6668246e9 2023-02-09 22:54:04,595 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/70f4c8c0df5d468f8551f4a54b07fe79 2023-02-09 22:54:04,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/57b08953287d4538b82e8dc2e61d17bf 2023-02-09 22:54:04,598 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/2393b0a2a8e54e35b839f758e647d8f0 2023-02-09 22:54:04,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a7bd3e7fca764acebab36fdf7d0e2b53 2023-02-09 22:54:04,601 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/c59ae7da41f74bd7a0d37b4dd9ae9148 2023-02-09 22:54:04,603 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/b627ca6096764a69808e95b105dc5cda_SeqId_44_ 2023-02-09 22:54:05,279 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231216398122322422dbbefaf23f6165493 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231216398122322422dbbefaf23f6165493 2023-02-09 22:54:05,280 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:05,312 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/377d53c0827c4619b5bee9d738d91b71 first=Optional[r1] last=Optional[r4] 2023-02-09 22:54:05,313 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/377d53c0827c4619b5bee9d738d91b71}] 2023-02-09 22:54:05,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/377d53c0827c4619b5bee9d738d91b71 for inclusion in de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:05,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 22:54:05,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:05,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:05,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/377d53c0827c4619b5bee9d738d91b71 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qbcgkki53in5pn4fkk3r8knfas6gok2o785b72p63al2knorf5ldlie81i2b8le1/family1/377d53c0827c4619b5bee9d738d91b71 2023-02-09 22:54:05,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qbcgkki53in5pn4fkk3r8knfas6gok2o785b72p63al2knorf5ldlie81i2b8le1/family1/377d53c0827c4619b5bee9d738d91b71 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_ 2023-02-09 22:54:05,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qbcgkki53in5pn4fkk3r8knfas6gok2o785b72p63al2knorf5ldlie81i2b8le1/family1/377d53c0827c4619b5bee9d738d91b71 into de6f52acd092b2a4a3ed68bb356a064e/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_ - updating store file list. 2023-02-09 22:54:05,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_ into de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:05,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qbcgkki53in5pn4fkk3r8knfas6gok2o785b72p63al2knorf5ldlie81i2b8le1/family1/377d53c0827c4619b5bee9d738d91b71 into de6f52acd092b2a4a3ed68bb356a064e/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_) 2023-02-09 22:54:05,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__qbcgkki53in5pn4fkk3r8knfas6gok2o785b72p63al2knorf5ldlie81i2b8le1/family1/377d53c0827c4619b5bee9d738d91b71 2023-02-09 22:54:05,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:54:05,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:54:05,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): de6f52acd092b2a4a3ed68bb356a064e/family1 is initiating minor compaction (all files) 2023-02-09 22:54:05,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7977bc55; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:05,333 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of de6f52acd092b2a4a3ed68bb356a064e/family1 in testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:54:05,333 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/16c9970bd6a3479285ea607798804d8a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp, totalSize=178.1 K 2023-02-09 22:54:05,334 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting a8c55c0456ce4cf7865377ae59d0b73a, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:54:05,335 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 16c9970bd6a3479285ea607798804d8a, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:54:05,335 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:05,335 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 9476d409715d4644897fe960bfc7fba7_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239414 2023-02-09 22:54:05,336 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:05,339 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120133b4717be73049878208d7bcd5bc6aa2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120133b4717be73049878208d7bcd5bc6aa2 2023-02-09 22:54:05,340 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205153d5d7452b742bf81c1f2e90081aafe to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205153d5d7452b742bf81c1f2e90081aafe 2023-02-09 22:54:05,342 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512282dc9cf52770e45558343ad2ce9f11a8a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512282dc9cf52770e45558343ad2ce9f11a8a 2023-02-09 22:54:05,344 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317a2142620de741ed9ccedab40b11faf3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512317a2142620de741ed9ccedab40b11faf3 2023-02-09 22:54:05,344 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 22:54:05,344 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 22:54:05,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:05,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:05,363 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): de6f52acd092b2a4a3ed68bb356a064e#family1#compaction#23 average throughput is 11.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:05,839 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113053788f80a16b44c0b2041ebb37b57308 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113053788f80a16b44c0b2041ebb37b57308 2023-02-09 22:54:05,840 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/.tmp/family1/e92165e6939f44e8b43af834e213c42a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/e92165e6939f44e8b43af834e213c42a 2023-02-09 22:54:05,840 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:05,847 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in de6f52acd092b2a4a3ed68bb356a064e/family1 of de6f52acd092b2a4a3ed68bb356a064e into e92165e6939f44e8b43af834e213c42a(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:05,847 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:05,848 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675983245333; duration=0sec 2023-02-09 22:54:05,848 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:05,870 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/716f68c7fe214a648ae1d43e7392ced3 first=Optional[r00] last=Optional[r8] 2023-02-09 22:54:05,870 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/716f68c7fe214a648ae1d43e7392ced3}] 2023-02-09 22:54:05,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/716f68c7fe214a648ae1d43e7392ced3 for inclusion in de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:05,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 22:54:05,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:05,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for de6f52acd092b2a4a3ed68bb356a064e: 2023-02-09 22:54:05,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/716f68c7fe214a648ae1d43e7392ced3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6la5ik3sh5v7ikcb611k74k551c1o1ffdhmogf3mpql5j52u31gb6am7s58n0t6u/family1/716f68c7fe214a648ae1d43e7392ced3 2023-02-09 22:54:05,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6la5ik3sh5v7ikcb611k74k551c1o1ffdhmogf3mpql5j52u31gb6am7s58n0t6u/family1/716f68c7fe214a648ae1d43e7392ced3 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a02c64b239a14f3183a1ef1e5650e239_SeqId_50_ 2023-02-09 22:54:05,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/16c9970bd6a3479285ea607798804d8a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_] to archive 2023-02-09 22:54:05,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6la5ik3sh5v7ikcb611k74k551c1o1ffdhmogf3mpql5j52u31gb6am7s58n0t6u/family1/716f68c7fe214a648ae1d43e7392ced3 into de6f52acd092b2a4a3ed68bb356a064e/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a02c64b239a14f3183a1ef1e5650e239_SeqId_50_ - updating store file list. 2023-02-09 22:54:05,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:05,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a8c55c0456ce4cf7865377ae59d0b73a 2023-02-09 22:54:05,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/16c9970bd6a3479285ea607798804d8a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/16c9970bd6a3479285ea607798804d8a 2023-02-09 22:54:05,889 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/9476d409715d4644897fe960bfc7fba7_SeqId_47_ 2023-02-09 22:54:05,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a02c64b239a14f3183a1ef1e5650e239_SeqId_50_ into de6f52acd092b2a4a3ed68bb356a064e/family1 2023-02-09 22:54:05,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6la5ik3sh5v7ikcb611k74k551c1o1ffdhmogf3mpql5j52u31gb6am7s58n0t6u/family1/716f68c7fe214a648ae1d43e7392ced3 into de6f52acd092b2a4a3ed68bb356a064e/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/family1/a02c64b239a14f3183a1ef1e5650e239_SeqId_50_) 2023-02-09 22:54:05,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6la5ik3sh5v7ikcb611k74k551c1o1ffdhmogf3mpql5j52u31gb6am7s58n0t6u/family1/716f68c7fe214a648ae1d43e7392ced3 2023-02-09 22:54:05,893 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:05,894 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:05,896 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120b0c8bdf674274bab9177459421955b84 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120b0c8bdf674274bab9177459421955b84 2023-02-09 22:54:05,897 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283b02b43ae7914fa6a0aa6fef22b9878d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283b02b43ae7914fa6a0aa6fef22b9878d 2023-02-09 22:54:05,899 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130fdbf772afacc45d49d594788e8a2e594 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130fdbf772afacc45d49d594788e8a2e594 2023-02-09 22:54:05,899 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 22:54:05,899 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 22:54:05,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:05,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:05,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:54:05,900 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:54:05,900 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:05,901 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:05,903 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.4160 sec 2023-02-09 22:54:05,998 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=256 (was 249) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43770 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43766 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43828 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43840 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43856 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-209665605_15 at /127.0.0.1:43778 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43868 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43904 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x66e9e9e9-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x1101648c-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x66e9e9e9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1888572195.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43986 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43742 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43814 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43710 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43926 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-209665605_15 at /127.0.0.1:43812 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:47990 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:59035@0x66e9e9e9-SendThread(127.0.0.1:59035) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43972 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1938397473_15 at /127.0.0.1:43912 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1101648c-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=615 (was 585) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 240), ProcessCount=168 (was 168), AvailableMemoryMB=11225 (was 11651) 2023-02-09 22:54:06,010 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256, OpenFileDescriptor=615, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=168, AvailableMemoryMB=11225 2023-02-09 22:54:06,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:54:06,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:06,015 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:06,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-09 22:54:06,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:06,016 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:06,018 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:06,019 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce empty. 2023-02-09 22:54:06,019 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:06,019 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-09 22:54:06,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:06,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:06,428 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:06,430 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => b9d6bd17e9df4ac20778cea4e1a554ce, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:06,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:06,836 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:06,837 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing b9d6bd17e9df4ac20778cea4e1a554ce, disabling compactions & flushes 2023-02-09 22:54:06,837 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:06,837 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:06,837 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. after waiting 0 ms 2023-02-09 22:54:06,837 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:06,837 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:06,837 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for b9d6bd17e9df4ac20778cea4e1a554ce: Waiting for close lock at 1675983246837Disabling compacts and flushes for region at 1675983246837Disabling writes for close at 1675983246837Writing region close event to WAL at 1675983246837Closed at 1675983246837 2023-02-09 22:54:06,840 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:06,842 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983246841"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983246841"}]},"ts":"1675983246841"} 2023-02-09 22:54:06,844 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:06,845 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:06,845 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983246845"}]},"ts":"1675983246845"} 2023-02-09 22:54:06,848 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 22:54:06,855 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b9d6bd17e9df4ac20778cea4e1a554ce, ASSIGN}] 2023-02-09 22:54:06,857 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b9d6bd17e9df4ac20778cea4e1a554ce, ASSIGN 2023-02-09 22:54:06,858 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b9d6bd17e9df4ac20778cea4e1a554ce, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:07,009 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=b9d6bd17e9df4ac20778cea4e1a554ce, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:07,010 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983247009"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983247009"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983247009"}]},"ts":"1675983247009"} 2023-02-09 22:54:07,012 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure b9d6bd17e9df4ac20778cea4e1a554ce, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:07,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:07,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:07,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b9d6bd17e9df4ac20778cea4e1a554ce, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:07,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:07,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,169 INFO [StoreOpener-b9d6bd17e9df4ac20778cea4e1a554ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,171 DEBUG [StoreOpener-b9d6bd17e9df4ac20778cea4e1a554ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:07,171 DEBUG [StoreOpener-b9d6bd17e9df4ac20778cea4e1a554ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:07,171 INFO [StoreOpener-b9d6bd17e9df4ac20778cea4e1a554ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9d6bd17e9df4ac20778cea4e1a554ce columnFamilyName family1 2023-02-09 22:54:07,172 INFO [StoreOpener-b9d6bd17e9df4ac20778cea4e1a554ce-1] regionserver.HStore(310): Store=b9d6bd17e9df4ac20778cea4e1a554ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:07,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:54:07,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:07,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened b9d6bd17e9df4ac20778cea4e1a554ce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68987292, jitterRate=0.027990758419036865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:07,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for b9d6bd17e9df4ac20778cea4e1a554ce: Running coprocessor pre-open hook at 1675983247168Writing region info on filesystem at 1675983247168Initializing all the Stores at 1675983247169 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983247169Cleaning up temporary data from old regions at 1675983247174 (+5 ms)Cleaning up detritus from prior splits at 1675983247174Running coprocessor post-open hooks at 1675983247179 (+5 ms)Region opened successfully at 1675983247179 2023-02-09 22:54:07,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce., pid=25, masterSystemTime=1675983247164 2023-02-09 22:54:07,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:07,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:07,182 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=b9d6bd17e9df4ac20778cea4e1a554ce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:07,182 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983247182"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983247182"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983247182"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983247182"}]},"ts":"1675983247182"} 2023-02-09 22:54:07,185 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-09 22:54:07,185 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure b9d6bd17e9df4ac20778cea4e1a554ce, server=jenkins-hbase9.apache.org,39093,1675983215026 in 172 msec 2023-02-09 22:54:07,187 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-09 22:54:07,187 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b9d6bd17e9df4ac20778cea4e1a554ce, ASSIGN in 330 msec 2023-02-09 22:54:07,188 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:07,188 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983247188"}]},"ts":"1675983247188"} 2023-02-09 22:54:07,189 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 22:54:07,193 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:07,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1810 sec 2023-02-09 22:54:08,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-09 22:54:08,122 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-09 22:54:08,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:08,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,146 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:08,146 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:08,146 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:08,146 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:08,146 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,146 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,147 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,147 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:08,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:08,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:08,148 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,148 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:08,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:08,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,149 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,149 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:08,149 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:08,149 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:08,149 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:08,149 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:08,149 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:08,150 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:08,151 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:54:08,180 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ad0ad55afb5c493e882dd1f225a30b90 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130ad0ad55afb5c493e882dd1f225a30b90 2023-02-09 22:54:08,184 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2e7d23bb6a5b46e58fbd336a3fb072c0 2023-02-09 22:54:08,197 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2e7d23bb6a5b46e58fbd336a3fb072c0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0 2023-02-09 22:54:08,207 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 22:54:08,209 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 58ms, sequenceid=5, compaction requested=false 2023-02-09 22:54:08,209 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:08,209 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:08,209 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:08,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:08,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:08,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:08,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:08,212 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,212 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:08,212 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,212 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@686acdc2[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:08,214 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:08,214 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,215 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:08,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:08,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:08,216 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,216 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:08,218 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:08,218 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,218 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:08,218 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:08,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:08,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:08,221 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:08,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:08,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:08,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,222 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:08,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:08,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:08,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:08,224 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:08,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,225 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,228 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,228 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:08,228 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,228 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:08,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:08,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:08,228 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:08,228 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:08,229 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:08,229 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:08,229 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:08,229 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:08,229 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:08,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:08,229 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,230 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,329 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:08,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:08,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:08,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:08,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,336 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:08,337 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:08,337 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:08,337 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:08,337 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,337 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,338 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:08,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:08,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:08,339 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:08,339 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,340 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:08,340 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:08,340 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:08,340 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:08,340 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:08,340 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:08,340 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:08,340 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:08,340 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:08,341 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:54:08,756 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ca9d76086521495597cb789bc14b5e34 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128ca9d76086521495597cb789bc14b5e34 2023-02-09 22:54:09,165 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/1cfbb192313548b2a63a6d3296438b1d 2023-02-09 22:54:09,172 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/1cfbb192313548b2a63a6d3296438b1d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d 2023-02-09 22:54:09,177 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:54:09,178 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 837ms, sequenceid=9, compaction requested=false 2023-02-09 22:54:09,179 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:09,179 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:09,179 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:09,179 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:09,179 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,179 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:09,179 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:09,181 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,181 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,181 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,182 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37049d77[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:09,183 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:09,183 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,184 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,184 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:09,184 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:09,184 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:09,184 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,185 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:09,186 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:09,186 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,186 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:09,189 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:09,189 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:09,189 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:09,189 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,189 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:09,191 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,191 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,191 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,192 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:09,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:09,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:09,195 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,195 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,196 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,196 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,295 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:09,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:09,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:09,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:09,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,304 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:09,304 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,304 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:09,304 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:09,305 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,305 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,306 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,306 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,307 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,307 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:09,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:09,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:09,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:09,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:09,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:09,308 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:09,308 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:09,309 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:09,346 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bfa9e327c60c497989c648d06439d3fa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfa9e327c60c497989c648d06439d3fa 2023-02-09 22:54:09,753 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2f90e614ef844852baabead4f1d60bbf 2023-02-09 22:54:09,759 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2f90e614ef844852baabead4f1d60bbf as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf 2023-02-09 22:54:09,763 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 22:54:09,764 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 455ms, sequenceid=13, compaction requested=true 2023-02-09 22:54:09,764 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:09,765 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:09,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:09,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:09,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:09,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:09,766 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,766 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,767 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,767 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62ce53e8[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:09,768 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:09,768 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,769 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:09,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:09,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:09,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:09,772 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:09,772 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,772 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,772 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:09,774 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:09,774 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:09,774 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:09,774 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,774 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:09,775 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:09,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,776 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,777 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:09,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:09,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:09,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:09,784 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:09,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:09,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:09,784 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,785 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,885 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:09,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:09,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:09,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:09,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,893 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:09,893 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,893 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:09,893 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:09,894 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,894 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,895 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,895 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:09,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:09,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:09,896 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:09,896 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:09,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:09,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,897 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:09,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:09,897 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:09,897 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:09,897 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:09,897 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:09,897 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:09,897 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:09,897 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:09,898 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:10,313 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f1b07afd41e9499586eef434bdb5e906 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1b07afd41e9499586eef434bdb5e906 2023-02-09 22:54:10,325 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/33a88e4c339c413093f0995dd42d3ab6 2023-02-09 22:54:10,330 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/33a88e4c339c413093f0995dd42d3ab6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6 2023-02-09 22:54:10,336 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 22:54:10,337 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 439ms, sequenceid=17, compaction requested=true 2023-02-09 22:54:10,337 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:10,337 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:10,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:10,339 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,339 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:10,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bcef983[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:10,341 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:10,341 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,342 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:10,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:10,343 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:10,343 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,343 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:10,345 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:10,345 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,345 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,345 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:10,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:10,349 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:10,349 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:10,349 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:10,349 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,349 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:10,351 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,351 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,351 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,353 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,355 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:10,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:10,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,356 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,456 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:10,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:10,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,463 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:10,463 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,463 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:10,463 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:10,463 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,463 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,464 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:10,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:10,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:10,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:10,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:10,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:10,467 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:10,467 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:10,467 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:10,485 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c07215bde157472fb9a59d3c84229daa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c07215bde157472fb9a59d3c84229daa 2023-02-09 22:54:10,491 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2855a8afaaec433480ea706198733971 2023-02-09 22:54:10,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/2855a8afaaec433480ea706198733971 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971 2023-02-09 22:54:10,504 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 22:54:10,505 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 38ms, sequenceid=21, compaction requested=true 2023-02-09 22:54:10,505 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:10,505 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:10,505 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:10,505 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:10,505 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,505 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:10,505 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:10,507 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,507 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,507 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,507 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:10,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55f42de5[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:10,509 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:10,509 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,510 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,510 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:10,510 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,510 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:10,510 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:10,510 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:10,512 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,512 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:10,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,513 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,513 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:10,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:10,516 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:10,516 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:10,516 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:10,516 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,516 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:10,518 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,518 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,518 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:10,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:10,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,518 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:10,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:10,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,523 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,523 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,524 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:10,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:10,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:10,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:10,523 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:10,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:10,524 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:10,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:10,625 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:10,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:10,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:10,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:10,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,631 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:10,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:10,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:10,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,633 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:10,633 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:10,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:10,634 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:10,634 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:10,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,634 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:10,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,634 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,634 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:10,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:10,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:10,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:10,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:10,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:10,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:10,635 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:10,635 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:10,635 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:11,048 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314b05041f195e4a318e7f06c4b610a219 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314b05041f195e4a318e7f06c4b610a219 2023-02-09 22:54:11,454 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/b6871e4e7d7542f997ffd2f36b5d4484 2023-02-09 22:54:11,460 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/b6871e4e7d7542f997ffd2f36b5d4484 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484 2023-02-09 22:54:11,464 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 22:54:11,465 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 830ms, sequenceid=25, compaction requested=true 2023-02-09 22:54:11,466 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:11,466 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:11,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:11,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:11,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:11,466 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:11,468 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:11,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:11,471 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:11,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4608ed09[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:11,471 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,472 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,472 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:11,472 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:11,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:11,472 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:11,474 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:11,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:11,474 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:11,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:11,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:11,477 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:11,477 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:11,477 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:11,477 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,477 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:11,482 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,482 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,482 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:11,483 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,484 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,486 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:11,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:11,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:11,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:11,487 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:11,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,487 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,488 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,488 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,587 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:11,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:11,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:11,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:11,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,594 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:11,594 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:11,594 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:11,594 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:11,595 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,595 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,596 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,596 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:11,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:11,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:11,596 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:11,596 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:11,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:11,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,597 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,597 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:11,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:11,597 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:11,597 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:11,598 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:11,598 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:11,598 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:11,598 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:11,598 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:11,598 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:12,013 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016fd5f393b5d8461f8e1550d103a46a87 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601016fd5f393b5d8461f8e1550d103a46a87 2023-02-09 22:54:12,421 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/8923391614d84324b0316e5b7ad40a24 2023-02-09 22:54:12,426 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/8923391614d84324b0316e5b7ad40a24 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24 2023-02-09 22:54:12,430 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 22:54:12,431 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 833ms, sequenceid=29, compaction requested=true 2023-02-09 22:54:12,431 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:12,431 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:12,431 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:12,431 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:12,431 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,432 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:12,432 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:12,433 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,433 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:12,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:12,433 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,433 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:12,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:12,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:12,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:12,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:12,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56c9a733[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:12,435 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:12,435 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,437 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,437 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:12,437 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,437 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:12,437 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:12,437 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:12,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:12,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:12,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:12,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:12,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:12,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:12,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:12,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:12,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:12,441 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:12,441 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:12,441 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:12,441 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,441 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:12,443 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:12,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:12,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:12,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:12,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,444 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,447 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:12,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:12,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:12,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:12,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:12,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:12,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:12,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:12,447 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:12,447 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:12,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:12,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:12,547 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:12,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:12,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:12,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:12,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,554 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:12,554 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:12,554 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:12,554 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:12,554 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,554 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,556 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,556 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:12,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:12,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:12,556 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,556 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:12,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,557 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:12,558 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:12,558 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:12,558 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:12,558 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:12,558 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:12,558 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:12,558 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:12,697 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:12,974 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ecba7d17eb4747bd8d319c307e18a27b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ecba7d17eb4747bd8d319c307e18a27b 2023-02-09 22:54:12,980 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/5a0f1aa89a2f484a84e5fcf6260b62a8 2023-02-09 22:54:12,985 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/5a0f1aa89a2f484a84e5fcf6260b62a8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8 2023-02-09 22:54:12,992 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 22:54:12,994 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 436ms, sequenceid=33, compaction requested=true 2023-02-09 22:54:12,994 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:12,994 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:12,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:12,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:12,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:12,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:12,996 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,996 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:12,996 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:12,996 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:12,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:12,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ad75e88[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:12,998 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:12,998 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:12,999 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:12,999 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:12,999 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:12,999 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:12,999 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:13,001 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:13,001 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,001 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:13,001 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:13,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:13,004 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:13,004 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:13,004 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:13,004 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,004 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:13,006 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,006 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:13,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,006 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:13,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:13,007 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,008 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:13,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:13,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:13,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:13,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:13,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,011 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:13,011 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:13,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:13,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:13,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:13,112 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:13,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:13,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:13,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:13,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,123 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:13,123 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:13,123 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:13,123 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:13,123 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,123 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,124 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:13,124 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,127 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,127 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:13,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,131 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:13,131 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:13,131 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:13,131 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:13,132 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:13,132 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:13,132 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:13,132 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:13,563 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078ac0915c470e48749eb13092e8f3c475 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509078ac0915c470e48749eb13092e8f3c475 2023-02-09 22:54:13,969 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/dedfcc2c4ac94d01bc12fe629b7b0d79 2023-02-09 22:54:13,974 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/dedfcc2c4ac94d01bc12fe629b7b0d79 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79 2023-02-09 22:54:13,979 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 22:54:13,980 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 848ms, sequenceid=37, compaction requested=true 2023-02-09 22:54:13,980 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:13,980 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:13,980 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:13,980 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:13,980 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,980 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:13,980 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:13,981 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,981 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:13,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:13,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:13,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:13,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:13,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:13,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:13,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eb8d932[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:13,983 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:13,983 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,989 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,989 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:13,989 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:13,989 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,989 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:13,989 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:13,990 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,990 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:13,990 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:13,991 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:13,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:13,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:13,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:13,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:13,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:13,993 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:13,993 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:13,993 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:13,993 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,993 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:13,994 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,994 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:13,994 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,995 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:13,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:13,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,996 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:13,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,998 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,998 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,998 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,999 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:13,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:13,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:13,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:13,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:13,999 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:13,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:14,099 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:14,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:14,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:14,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:14,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,106 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:14,106 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:14,106 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:14,107 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:14,107 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,107 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,110 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,110 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:14,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:14,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:14,110 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,110 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:14,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,110 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,111 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,111 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:14,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,111 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:14,111 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:14,111 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:14,111 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:14,111 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:14,111 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:14,112 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. started... 2023-02-09 22:54:14,112 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b9d6bd17e9df4ac20778cea4e1a554ce 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:14,538 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511201a6437010ea74f7fbec29b9382434f7b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511201a6437010ea74f7fbec29b9382434f7b 2023-02-09 22:54:14,945 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/4a20c910d614416c9a151633d85fc80d 2023-02-09 22:54:14,952 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/4a20c910d614416c9a151633d85fc80d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d 2023-02-09 22:54:14,958 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 22:54:14,959 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b9d6bd17e9df4ac20778cea4e1a554ce in 847ms, sequenceid=41, compaction requested=true 2023-02-09 22:54:14,959 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:14,959 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:14,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:14,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:14,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:14,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:14,961 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,961 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:14,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:14,961 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,961 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:14,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:14,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:14,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:14,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:14,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fd91b49[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:14,963 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:14,963 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,964 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,964 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:14,964 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:14,964 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,964 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:14,964 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:14,966 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:14,966 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,966 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:14,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,966 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:14,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:14,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:14,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:14,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:14,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:14,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:14,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:14,969 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:14,970 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:14,970 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:14,970 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,970 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:14,972 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,972 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:14,972 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:14,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:14,973 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:14,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,974 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:14,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-09 22:54:14,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:14,978 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:14,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:14,978 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:14,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:14,979 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:15,079 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:15,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:15,081 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-09 22:54:15,083 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:15,084 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:15,085 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983855085 2023-02-09 22:54:15,085 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:15,086 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:54:15,088 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 22:54:15,088 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:15,088 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x4ab1c0f8 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:15,095 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bbdf847, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:15,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,538 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032a9fbbab45604b29a0561f6ce7769171 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601032a9fbbab45604b29a0561f6ce7769171 2023-02-09 22:54:15,539 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:15,553 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:15,554 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:55052, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:15,562 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:15,563 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:53824, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:15,574 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a56a6e3f587e41cb8a5d03e76f2888dc first=Optional[r3] last=Optional[r6] 2023-02-09 22:54:15,575 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a56a6e3f587e41cb8a5d03e76f2888dc}] 2023-02-09 22:54:15,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a56a6e3f587e41cb8a5d03e76f2888dc for inclusion in b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:15,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 22:54:15,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:15,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:15,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a56a6e3f587e41cb8a5d03e76f2888dc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__s3e1hmoe2t7qav3ki727o1kirnb36196naghl2kqre0rmraglm3cea7t4ej1cei8/family1/a56a6e3f587e41cb8a5d03e76f2888dc 2023-02-09 22:54:15,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__s3e1hmoe2t7qav3ki727o1kirnb36196naghl2kqre0rmraglm3cea7t4ej1cei8/family1/a56a6e3f587e41cb8a5d03e76f2888dc as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_ 2023-02-09 22:54:15,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__s3e1hmoe2t7qav3ki727o1kirnb36196naghl2kqre0rmraglm3cea7t4ej1cei8/family1/a56a6e3f587e41cb8a5d03e76f2888dc into b9d6bd17e9df4ac20778cea4e1a554ce/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_ - updating store file list. 2023-02-09 22:54:15,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_ into b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:15,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__s3e1hmoe2t7qav3ki727o1kirnb36196naghl2kqre0rmraglm3cea7t4ej1cei8/family1/a56a6e3f587e41cb8a5d03e76f2888dc into b9d6bd17e9df4ac20778cea4e1a554ce/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_) 2023-02-09 22:54:15,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__s3e1hmoe2t7qav3ki727o1kirnb36196naghl2kqre0rmraglm3cea7t4ej1cei8/family1/a56a6e3f587e41cb8a5d03e76f2888dc 2023-02-09 22:54:15,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:54:15,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:54:15,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): b9d6bd17e9df4ac20778cea4e1a554ce/family1 is initiating minor compaction 2023-02-09 22:54:15,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@45b96dbb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:15,597 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b9d6bd17e9df4ac20778cea4e1a554ce/family1 in testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:15,598 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp, totalSize=50.0 K 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 1cfbb192313548b2a63a6d3296438b1d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2f90e614ef844852baabead4f1d60bbf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 33a88e4c339c413093f0995dd42d3ab6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2855a8afaaec433480ea706198733971, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b6871e4e7d7542f997ffd2f36b5d4484, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 8923391614d84324b0316e5b7ad40a24, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 5a0f1aa89a2f484a84e5fcf6260b62a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting dedfcc2c4ac94d01bc12fe629b7b0d79, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 4a20c910d614416c9a151633d85fc80d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:54:15,598 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b094e54fe1194eda8abe103b0cdf798e_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:54:15,600 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:15,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:15,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c07215bde157472fb9a59d3c84229daa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c07215bde157472fb9a59d3c84229daa 2023-02-09 22:54:15,604 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314b05041f195e4a318e7f06c4b610a219 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314b05041f195e4a318e7f06c4b610a219 2023-02-09 22:54:15,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601016fd5f393b5d8461f8e1550d103a46a87 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601016fd5f393b5d8461f8e1550d103a46a87 2023-02-09 22:54:15,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ecba7d17eb4747bd8d319c307e18a27b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ecba7d17eb4747bd8d319c307e18a27b 2023-02-09 22:54:15,607 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 22:54:15,607 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 22:54:15,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,607 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:15,633 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): b9d6bd17e9df4ac20778cea4e1a554ce#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:16,045 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120567576af9a4014ac7bb5f906c70e37803 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120567576af9a4014ac7bb5f906c70e37803 2023-02-09 22:54:16,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:16,053 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/071be32ee9464f53a48fd2fbdb389ee4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/071be32ee9464f53a48fd2fbdb389ee4 2023-02-09 22:54:16,063 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b9d6bd17e9df4ac20778cea4e1a554ce/family1 of b9d6bd17e9df4ac20778cea4e1a554ce into 071be32ee9464f53a48fd2fbdb389ee4(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:16,063 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:16,064 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675983255597; duration=0sec 2023-02-09 22:54:16,064 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:16,080 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cccb8bc0e3a04dcfa221681fe1b66e02 first=Optional[r1] last=Optional[r2] 2023-02-09 22:54:16,080 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cccb8bc0e3a04dcfa221681fe1b66e02}] 2023-02-09 22:54:16,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cccb8bc0e3a04dcfa221681fe1b66e02 for inclusion in b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:16,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_] to archive 2023-02-09 22:54:16,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:16,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 22:54:16,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:16,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:16,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cccb8bc0e3a04dcfa221681fe1b66e02 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__kfteis72krvp7a18520cjb8485m9bb4nn74pp30hlrp45fl54415sikjt0ub8k56/family1/cccb8bc0e3a04dcfa221681fe1b66e02 2023-02-09 22:54:16,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/1cfbb192313548b2a63a6d3296438b1d 2023-02-09 22:54:16,100 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2f90e614ef844852baabead4f1d60bbf 2023-02-09 22:54:16,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__kfteis72krvp7a18520cjb8485m9bb4nn74pp30hlrp45fl54415sikjt0ub8k56/family1/cccb8bc0e3a04dcfa221681fe1b66e02 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_ 2023-02-09 22:54:16,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__kfteis72krvp7a18520cjb8485m9bb4nn74pp30hlrp45fl54415sikjt0ub8k56/family1/cccb8bc0e3a04dcfa221681fe1b66e02 into b9d6bd17e9df4ac20778cea4e1a554ce/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_ - updating store file list. 2023-02-09 22:54:16,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/33a88e4c339c413093f0995dd42d3ab6 2023-02-09 22:54:16,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2855a8afaaec433480ea706198733971 2023-02-09 22:54:16,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b6871e4e7d7542f997ffd2f36b5d4484 2023-02-09 22:54:16,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_ into b9d6bd17e9df4ac20778cea4e1a554ce/family1 2023-02-09 22:54:16,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__kfteis72krvp7a18520cjb8485m9bb4nn74pp30hlrp45fl54415sikjt0ub8k56/family1/cccb8bc0e3a04dcfa221681fe1b66e02 into b9d6bd17e9df4ac20778cea4e1a554ce/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_) 2023-02-09 22:54:16,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/8923391614d84324b0316e5b7ad40a24 2023-02-09 22:54:16,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithWeeklyPolicy__kfteis72krvp7a18520cjb8485m9bb4nn74pp30hlrp45fl54415sikjt0ub8k56/family1/cccb8bc0e3a04dcfa221681fe1b66e02 2023-02-09 22:54:16,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:54:16,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/5a0f1aa89a2f484a84e5fcf6260b62a8 2023-02-09 22:54:16,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:54:16,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): b9d6bd17e9df4ac20778cea4e1a554ce/family1 is initiating minor compaction (all files) 2023-02-09 22:54:16,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f917438; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:16,108 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b9d6bd17e9df4ac20778cea4e1a554ce/family1 in testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:54:16,109 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/071be32ee9464f53a48fd2fbdb389ee4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp, totalSize=176.8 K 2023-02-09 22:54:16,109 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2e7d23bb6a5b46e58fbd336a3fb072c0, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:54:16,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/dedfcc2c4ac94d01bc12fe629b7b0d79 2023-02-09 22:54:16,110 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 071be32ee9464f53a48fd2fbdb389ee4, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:54:16,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:16,111 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 7567a65ce9e64ae288851c102f0ca014_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965239414 2023-02-09 22:54:16,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/4a20c910d614416c9a151633d85fc80d 2023-02-09 22:54:16,112 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:16,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/b094e54fe1194eda8abe103b0cdf798e_SeqId_44_ 2023-02-09 22:54:16,113 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfa9e327c60c497989c648d06439d3fa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfa9e327c60c497989c648d06439d3fa 2023-02-09 22:54:16,115 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1b07afd41e9499586eef434bdb5e906 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1b07afd41e9499586eef434bdb5e906 2023-02-09 22:54:16,115 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-09 22:54:16,115 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 22:54:16,116 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:16,116 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:16,116 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:16,117 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:16,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0350 sec 2023-02-09 22:54:16,133 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): b9d6bd17e9df4ac20778cea4e1a554ce#family1#compaction#35 average throughput is 14.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:16,254 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=261 (was 256) - Thread LEAK? -, OpenFileDescriptor=617 (was 615) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=222 (was 210) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10997 (was 11225) 2023-02-09 22:54:16,269 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=261, OpenFileDescriptor=617, MaxFileDescriptor=60000, SystemLoadAverage=222, ProcessCount=168, AvailableMemoryMB=10997 2023-02-09 22:54:16,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 22:54:16,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:16,277 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:16,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-09 22:54:16,278 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:16,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:16,281 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:16,281 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:16,282 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d empty. 2023-02-09 22:54:16,282 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:16,283 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:16,283 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb empty. 2023-02-09 22:54:16,283 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4 empty. 2023-02-09 22:54:16,284 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:16,284 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:16,284 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-09 22:54:16,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:16,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:16,694 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:16,696 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 99b4221c7574c0df4a7deef1f67b4db4, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:16,697 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => b666c69c9d5408957536862638a47fdb, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:16,697 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => dc942a927e63be0b0db6ded05de4001d, NAME => 'testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:16,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:16,983 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/.tmp/family1/e31dd8501ac343b6be2768c159e7807f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/e31dd8501ac343b6be2768c159e7807f 2023-02-09 22:54:16,989 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b9d6bd17e9df4ac20778cea4e1a554ce/family1 of b9d6bd17e9df4ac20778cea4e1a554ce into e31dd8501ac343b6be2768c159e7807f(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:16,989 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b9d6bd17e9df4ac20778cea4e1a554ce: 2023-02-09 22:54:16,990 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1675983256108; duration=0sec 2023-02-09 22:54:16,990 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:17,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/071be32ee9464f53a48fd2fbdb389ee4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_] to archive 2023-02-09 22:54:17,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:17,086 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/2e7d23bb6a5b46e58fbd336a3fb072c0 2023-02-09 22:54:17,087 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/071be32ee9464f53a48fd2fbdb389ee4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/071be32ee9464f53a48fd2fbdb389ee4 2023-02-09 22:54:17,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/family1/7567a65ce9e64ae288851c102f0ca014_SeqId_47_ 2023-02-09 22:54:17,104 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing dc942a927e63be0b0db6ded05de4001d, disabling compactions & flushes 2023-02-09 22:54:17,105 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. after waiting 0 ms 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,105 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for dc942a927e63be0b0db6ded05de4001d: Waiting for close lock at 1675983257105Disabling compacts and flushes for region at 1675983257105Disabling writes for close at 1675983257105Writing region close event to WAL at 1675983257105Closed at 1675983257105 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing b666c69c9d5408957536862638a47fdb, disabling compactions & flushes 2023-02-09 22:54:17,105 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. after waiting 0 ms 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,105 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,105 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for b666c69c9d5408957536862638a47fdb: Waiting for close lock at 1675983257105Disabling compacts and flushes for region at 1675983257105Disabling writes for close at 1675983257105Writing region close event to WAL at 1675983257105Closed at 1675983257105 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 99b4221c7574c0df4a7deef1f67b4db4, disabling compactions & flushes 2023-02-09 22:54:17,109 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. after waiting 0 ms 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,109 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,109 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 99b4221c7574c0df4a7deef1f67b4db4: Waiting for close lock at 1675983257109Disabling compacts and flushes for region at 1675983257109Disabling writes for close at 1675983257109Writing region close event to WAL at 1675983257109Closed at 1675983257109 2023-02-09 22:54:17,112 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:17,113 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983257113"}]},"ts":"1675983257113"} 2023-02-09 22:54:17,113 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983257113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983257113"}]},"ts":"1675983257113"} 2023-02-09 22:54:17,113 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257113"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983257113"}]},"ts":"1675983257113"} 2023-02-09 22:54:17,116 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 22:54:17,117 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:17,117 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983257117"}]},"ts":"1675983257117"} 2023-02-09 22:54:17,118 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-09 22:54:17,121 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=dc942a927e63be0b0db6ded05de4001d, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b666c69c9d5408957536862638a47fdb, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=99b4221c7574c0df4a7deef1f67b4db4, ASSIGN}] 2023-02-09 22:54:17,123 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b666c69c9d5408957536862638a47fdb, ASSIGN 2023-02-09 22:54:17,123 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=dc942a927e63be0b0db6ded05de4001d, ASSIGN 2023-02-09 22:54:17,123 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=99b4221c7574c0df4a7deef1f67b4db4, ASSIGN 2023-02-09 22:54:17,123 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b666c69c9d5408957536862638a47fdb, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:17,123 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=dc942a927e63be0b0db6ded05de4001d, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:17,124 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=99b4221c7574c0df4a7deef1f67b4db4, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:17,275 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=99b4221c7574c0df4a7deef1f67b4db4, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,275 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=dc942a927e63be0b0db6ded05de4001d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,275 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b666c69c9d5408957536862638a47fdb, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,275 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983257275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983257275"}]},"ts":"1675983257275"} 2023-02-09 22:54:17,275 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983257275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983257275"}]},"ts":"1675983257275"} 2023-02-09 22:54:17,275 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983257275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983257275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983257275"}]},"ts":"1675983257275"} 2023-02-09 22:54:17,277 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 99b4221c7574c0df4a7deef1f67b4db4, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:17,278 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure dc942a927e63be0b0db6ded05de4001d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:17,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure b666c69c9d5408957536862638a47fdb, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:17,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:17,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b666c69c9d5408957536862638a47fdb, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 22:54:17,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,434 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,436 DEBUG [StoreOpener-b666c69c9d5408957536862638a47fdb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/family1 2023-02-09 22:54:17,436 DEBUG [StoreOpener-b666c69c9d5408957536862638a47fdb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/family1 2023-02-09 22:54:17,436 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b666c69c9d5408957536862638a47fdb columnFamilyName family1 2023-02-09 22:54:17,438 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] regionserver.HStore(310): Store=b666c69c9d5408957536862638a47fdb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,438 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,440 DEBUG [StoreOpener-b666c69c9d5408957536862638a47fdb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/family2 2023-02-09 22:54:17,440 DEBUG [StoreOpener-b666c69c9d5408957536862638a47fdb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/family2 2023-02-09 22:54:17,440 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b666c69c9d5408957536862638a47fdb columnFamilyName family2 2023-02-09 22:54:17,442 INFO [StoreOpener-b666c69c9d5408957536862638a47fdb-1] regionserver.HStore(310): Store=b666c69c9d5408957536862638a47fdb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:17,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for b666c69c9d5408957536862638a47fdb 2023-02-09 22:54:17,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:17,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened b666c69c9d5408957536862638a47fdb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66048736, jitterRate=-0.015797138214111328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:17,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for b666c69c9d5408957536862638a47fdb: Running coprocessor pre-open hook at 1675983257433Writing region info on filesystem at 1675983257433Initializing all the Stores at 1675983257434 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257434Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257434Cleaning up temporary data from old regions at 1675983257444 (+10 ms)Cleaning up detritus from prior splits at 1675983257444Running coprocessor post-open hooks at 1675983257449 (+5 ms)Region opened successfully at 1675983257449 2023-02-09 22:54:17,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb., pid=33, masterSystemTime=1675983257429 2023-02-09 22:54:17,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:17,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dc942a927e63be0b0db6ded05de4001d, NAME => 'testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 22:54:17,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,452 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b666c69c9d5408957536862638a47fdb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,453 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675983257452"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983257452"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983257452"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983257452"}]},"ts":"1675983257452"} 2023-02-09 22:54:17,454 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,455 DEBUG [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/family1 2023-02-09 22:54:17,455 DEBUG [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/family1 2023-02-09 22:54:17,456 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc942a927e63be0b0db6ded05de4001d columnFamilyName family1 2023-02-09 22:54:17,456 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-09 22:54:17,456 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure b666c69c9d5408957536862638a47fdb, server=jenkins-hbase9.apache.org,39093,1675983215026 in 175 msec 2023-02-09 22:54:17,457 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] regionserver.HStore(310): Store=dc942a927e63be0b0db6ded05de4001d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,457 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b666c69c9d5408957536862638a47fdb, ASSIGN in 335 msec 2023-02-09 22:54:17,458 DEBUG [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/family2 2023-02-09 22:54:17,458 DEBUG [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/family2 2023-02-09 22:54:17,459 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc942a927e63be0b0db6ded05de4001d columnFamilyName family2 2023-02-09 22:54:17,460 INFO [StoreOpener-dc942a927e63be0b0db6ded05de4001d-1] regionserver.HStore(310): Store=dc942a927e63be0b0db6ded05de4001d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:17,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:54:17,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:17,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened dc942a927e63be0b0db6ded05de4001d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61162680, jitterRate=-0.08860504627227783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:17,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for dc942a927e63be0b0db6ded05de4001d: Running coprocessor pre-open hook at 1675983257453Writing region info on filesystem at 1675983257453Initializing all the Stores at 1675983257453Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257454 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257454Cleaning up temporary data from old regions at 1675983257462 (+8 ms)Cleaning up detritus from prior splits at 1675983257462Running coprocessor post-open hooks at 1675983257466 (+4 ms)Region opened successfully at 1675983257466 2023-02-09 22:54:17,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d., pid=32, masterSystemTime=1675983257429 2023-02-09 22:54:17,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:17,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 99b4221c7574c0df4a7deef1f67b4db4, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 22:54:17,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,469 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=dc942a927e63be0b0db6ded05de4001d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:17,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,469 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257469"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983257469"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983257469"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983257469"}]},"ts":"1675983257469"} 2023-02-09 22:54:17,470 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,471 DEBUG [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1 2023-02-09 22:54:17,471 DEBUG [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1 2023-02-09 22:54:17,472 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99b4221c7574c0df4a7deef1f67b4db4 columnFamilyName family1 2023-02-09 22:54:17,472 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-09 22:54:17,472 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure dc942a927e63be0b0db6ded05de4001d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 192 msec 2023-02-09 22:54:17,473 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] regionserver.HStore(310): Store=99b4221c7574c0df4a7deef1f67b4db4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,473 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,473 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=dc942a927e63be0b0db6ded05de4001d, ASSIGN in 351 msec 2023-02-09 22:54:17,474 DEBUG [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family2 2023-02-09 22:54:17,474 DEBUG [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family2 2023-02-09 22:54:17,475 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99b4221c7574c0df4a7deef1f67b4db4 columnFamilyName family2 2023-02-09 22:54:17,476 INFO [StoreOpener-99b4221c7574c0df4a7deef1f67b4db4-1] regionserver.HStore(310): Store=99b4221c7574c0df4a7deef1f67b4db4/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:17,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:54:17,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:54:17,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:17,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 99b4221c7574c0df4a7deef1f67b4db4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64132625, jitterRate=-0.044349417090415955}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:54:17,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 99b4221c7574c0df4a7deef1f67b4db4: Running coprocessor pre-open hook at 1675983257469Writing region info on filesystem at 1675983257469Initializing all the Stores at 1675983257470 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257470Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983257470Cleaning up temporary data from old regions at 1675983257477 (+7 ms)Cleaning up detritus from prior splits at 1675983257478 (+1 ms)Running coprocessor post-open hooks at 1675983257481 (+3 ms)Region opened successfully at 1675983257481 2023-02-09 22:54:17,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4., pid=31, masterSystemTime=1675983257429 2023-02-09 22:54:17,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:17,484 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=99b4221c7574c0df4a7deef1f67b4db4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:17,484 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675983257484"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983257484"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983257484"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983257484"}]},"ts":"1675983257484"} 2023-02-09 22:54:17,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-02-09 22:54:17,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 99b4221c7574c0df4a7deef1f67b4db4, server=jenkins-hbase9.apache.org,39093,1675983215026 in 208 msec 2023-02-09 22:54:17,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-02-09 22:54:17,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=99b4221c7574c0df4a7deef1f67b4db4, ASSIGN in 366 msec 2023-02-09 22:54:17,490 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:17,490 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983257490"}]},"ts":"1675983257490"} 2023-02-09 22:54:17,492 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-09 22:54:17,494 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:17,517 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:17,517 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66e9e9e9 to 127.0.0.1:59035 2023-02-09 22:54:17,517 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:17,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2210 sec 2023-02-09 22:54:17,523 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:17,523 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37261d86 to 127.0.0.1:59035 2023-02-09 22:54:17,523 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:17,526 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:17,526 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x688a6e25 to 127.0.0.1:59035 2023-02-09 22:54:17,526 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:17,526 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:17,527 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ab1c0f8 to 127.0.0.1:59035 2023-02-09 22:54:17,527 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:18,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-09 22:54:18,385 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-09 22:54:18,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:18,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,394 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 22:54:18,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:18,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 22:54:18,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 22:54:18,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,397 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:18,397 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:18,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:18,397 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:18,397 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:18,398 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 22:54:18,398 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:18,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 22:54:18,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:18,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. started... 2023-02-09 22:54:18,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:18,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. started... 2023-02-09 22:54:18,403 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:54:18,403 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b666c69c9d5408957536862638a47fdb: 2023-02-09 22:54:18,403 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:18,403 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:18,403 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 99b4221c7574c0df4a7deef1f67b4db4 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-09 22:54:18,403 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. started... 2023-02-09 22:54:18,403 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:18,404 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:54:18,404 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for dc942a927e63be0b0db6ded05de4001d: 2023-02-09 22:54:18,404 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:18,404 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:54:18,820 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209fabf20bd5e38473f94a36916d4aea6ad to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fabf20bd5e38473f94a36916d4aea6ad 2023-02-09 22:54:18,941 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-09 22:54:19,225 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp/family1/67fa62ac74ad4658a851dd39823742b8 2023-02-09 22:54:19,231 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp/family1/67fa62ac74ad4658a851dd39823742b8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8 2023-02-09 22:54:19,236 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 22:54:19,237 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 99b4221c7574c0df4a7deef1f67b4db4 in 834ms, sequenceid=5, compaction requested=false 2023-02-09 22:54:19,238 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 99b4221c7574c0df4a7deef1f67b4db4: 2023-02-09 22:54:19,238 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:19,238 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:54:19,238 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:54:19,238 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,238 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 22:54:19,238 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 22:54:19,240 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,240 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,240 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,240 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:19,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 22:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24b2d3be[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:19,242 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 22:54:19,242 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,243 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,243 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 22:54:19,243 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 22:54:19,243 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:19,243 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,243 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:19,245 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 22:54:19,245 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,245 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,245 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 22:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:19,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 22:54:19,247 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 22:54:19,247 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:19,247 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:19,247 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,247 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:19,250 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,250 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,250 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,251 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,252 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,261 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:19,261 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:19,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:19,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:19,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 22:54:19,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:19,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,263 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,363 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:19,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:19,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:19,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:19,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,372 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-09 22:54:19,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:19,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-09 22:54:19,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-09 22:54:19,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,374 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:19,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:19,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:19,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:19,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:19,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-09 22:54:19,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-09 22:54:19,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:19,379 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-09 22:54:19,379 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:19,380 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. started... 2023-02-09 22:54:19,380 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b666c69c9d5408957536862638a47fdb: 2023-02-09 22:54:19,380 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:54:19,380 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:19,380 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:19,380 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:54:19,380 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. started... 2023-02-09 22:54:19,380 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 99b4221c7574c0df4a7deef1f67b4db4 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-09 22:54:19,380 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:54:19,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:19,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. started... 2023-02-09 22:54:19,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for dc942a927e63be0b0db6ded05de4001d: 2023-02-09 22:54:19,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:54:19,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:54:19,414 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209cc6d23850e8a4c48b4d2c90112fcbda5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc6d23850e8a4c48b4d2c90112fcbda5 2023-02-09 22:54:19,425 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp/family1/34940a653a834c4d8629a6e18b9f5873 2023-02-09 22:54:19,430 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp/family1/34940a653a834c4d8629a6e18b9f5873 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873 2023-02-09 22:54:19,434 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873, entries=2, sequenceid=9, filesize=5.0 K 2023-02-09 22:54:19,435 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 99b4221c7574c0df4a7deef1f67b4db4 in 55ms, sequenceid=9, compaction requested=false 2023-02-09 22:54:19,435 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 99b4221c7574c0df4a7deef1f67b4db4: 2023-02-09 22:54:19,435 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:19,435 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:54:19,435 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:54:19,435 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,435 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-09 22:54:19,435 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-09 22:54:19,437 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,437 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:19,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-09 22:54:19,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54e1fabe[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:19,440 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-09 22:54:19,440 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,441 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-09 22:54:19,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-09 22:54:19,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:19,441 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,441 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:19,443 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,443 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-09 22:54:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,443 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:19,443 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-09 22:54:19,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:19,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-09 22:54:19,445 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-09 22:54:19,445 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:19,445 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:19,445 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,445 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:19,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:19,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:19,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:19,448 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:19,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,449 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:19,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,453 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,453 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-09 22:54:19,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:19,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:19,453 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:19,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-09 22:54:19,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:19,453 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:19,453 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:19,554 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:19,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:19,561 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-09 22:54:19,563 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 22:54:19,563 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 22:54:19,564 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:19,564 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983859564 2023-02-09 22:54:19,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:19,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-09 22:54:19,565 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 22:54:19,566 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x0085e7ca to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:19,570 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@245e485d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:19,570 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 22:54:20,397 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302090b56f80c37e744c1bf27086458f35b07 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302090b56f80c37e744c1bf27086458f35b07 2023-02-09 22:54:20,398 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 22:54:20,412 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:20,413 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:39204, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:20,420 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:20,422 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:56316, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:20,434 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5be06006567f4d61b779e72447ee7d55 first=Optional[k0] last=Optional[k1] 2023-02-09 22:54:20,434 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5be06006567f4d61b779e72447ee7d55}] 2023-02-09 22:54:20,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5be06006567f4d61b779e72447ee7d55 for inclusion in 99b4221c7574c0df4a7deef1f67b4db4/family1 2023-02-09 22:54:20,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-09 22:54:20,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 22:54:20,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 99b4221c7574c0df4a7deef1f67b4db4: 2023-02-09 22:54:20,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5be06006567f4d61b779e72447ee7d55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testScannerOnBulkLoadRefHFiles__h7dhavqelidurcct4nqp223kbhfbmhtua9mttsfovdeq9pnnsibo8na5o83njrlv/family1/5be06006567f4d61b779e72447ee7d55 2023-02-09 22:54:20,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testScannerOnBulkLoadRefHFiles__h7dhavqelidurcct4nqp223kbhfbmhtua9mttsfovdeq9pnnsibo8na5o83njrlv/family1/5be06006567f4d61b779e72447ee7d55 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_ 2023-02-09 22:54:20,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testScannerOnBulkLoadRefHFiles__h7dhavqelidurcct4nqp223kbhfbmhtua9mttsfovdeq9pnnsibo8na5o83njrlv/family1/5be06006567f4d61b779e72447ee7d55 into 99b4221c7574c0df4a7deef1f67b4db4/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_ - updating store file list. 2023-02-09 22:54:20,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_ into 99b4221c7574c0df4a7deef1f67b4db4/family1 2023-02-09 22:54:20,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testScannerOnBulkLoadRefHFiles__h7dhavqelidurcct4nqp223kbhfbmhtua9mttsfovdeq9pnnsibo8na5o83njrlv/family1/5be06006567f4d61b779e72447ee7d55 into 99b4221c7574c0df4a7deef1f67b4db4/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_) 2023-02-09 22:54:20,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testScannerOnBulkLoadRefHFiles__h7dhavqelidurcct4nqp223kbhfbmhtua9mttsfovdeq9pnnsibo8na5o83njrlv/family1/5be06006567f4d61b779e72447ee7d55 2023-02-09 22:54:20,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:54:20,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:54:20,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): 99b4221c7574c0df4a7deef1f67b4db4/family1 is initiating minor compaction (all files) 2023-02-09 22:54:20,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2ce08def; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:20,455 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 99b4221c7574c0df4a7deef1f67b4db4/family1 in testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:54:20,455 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp, totalSize=15.1 K 2023-02-09 22:54:20,456 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 67fa62ac74ad4658a851dd39823742b8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983258385 2023-02-09 22:54:20,456 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 34940a653a834c4d8629a6e18b9f5873, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983258385 2023-02-09 22:54:20,457 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 052dfe1474a84fdc805d0064d3a77f50_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983258385 2023-02-09 22:54:20,457 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 22:54:20,458 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:20,460 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc6d23850e8a4c48b4d2c90112fcbda5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209cc6d23850e8a4c48b4d2c90112fcbda5 2023-02-09 22:54:20,461 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fabf20bd5e38473f94a36916d4aea6ad to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230209fabf20bd5e38473f94a36916d4aea6ad 2023-02-09 22:54:20,461 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-09 22:54:20,461 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 22:54:20,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-09 22:54:20,462 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-09 22:54:20,462 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:20,463 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-09 22:54:20,465 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 901 msec 2023-02-09 22:54:20,467 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 99b4221c7574c0df4a7deef1f67b4db4#family1#compaction#38 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:20,492 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=249 (was 261), OpenFileDescriptor=582 (was 617), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 222) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10935 (was 10997) 2023-02-09 22:54:20,503 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=249, OpenFileDescriptor=582, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=168, AvailableMemoryMB=10935 2023-02-09 22:54:20,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:54:20,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:20,506 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:20,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-09 22:54:20,507 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:20,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:20,508 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:20,509 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826 empty. 2023-02-09 22:54:20,509 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:20,509 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-09 22:54:20,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:20,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:20,917 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:20,918 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1f9ad9d3e00d7401d4d1052fa2e47826, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:21,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:21,286 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/.tmp/family1/200a3bdd2e464df6986875b925ecc85b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/200a3bdd2e464df6986875b925ecc85b 2023-02-09 22:54:21,290 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 99b4221c7574c0df4a7deef1f67b4db4/family1 of 99b4221c7574c0df4a7deef1f67b4db4 into 200a3bdd2e464df6986875b925ecc85b(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:21,290 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 99b4221c7574c0df4a7deef1f67b4db4: 2023-02-09 22:54:21,290 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1675983260455; duration=0sec 2023-02-09 22:54:21,290 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:21,324 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:21,324 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 1f9ad9d3e00d7401d4d1052fa2e47826, disabling compactions & flushes 2023-02-09 22:54:21,325 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,325 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,325 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. after waiting 0 ms 2023-02-09 22:54:21,325 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,325 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,325 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 1f9ad9d3e00d7401d4d1052fa2e47826: Waiting for close lock at 1675983261324Disabling compacts and flushes for region at 1675983261324Disabling writes for close at 1675983261325 (+1 ms)Writing region close event to WAL at 1675983261325Closed at 1675983261325 2023-02-09 22:54:21,327 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:21,328 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983261328"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983261328"}]},"ts":"1675983261328"} 2023-02-09 22:54:21,330 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:21,331 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:21,331 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983261331"}]},"ts":"1675983261331"} 2023-02-09 22:54:21,332 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-09 22:54:21,335 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=1f9ad9d3e00d7401d4d1052fa2e47826, ASSIGN}] 2023-02-09 22:54:21,337 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=1f9ad9d3e00d7401d4d1052fa2e47826, ASSIGN 2023-02-09 22:54:21,337 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=1f9ad9d3e00d7401d4d1052fa2e47826, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:21,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_] to archive 2023-02-09 22:54:21,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:21,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/67fa62ac74ad4658a851dd39823742b8 2023-02-09 22:54:21,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/34940a653a834c4d8629a6e18b9f5873 2023-02-09 22:54:21,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/family1/052dfe1474a84fdc805d0064d3a77f50_SeqId_12_ 2023-02-09 22:54:21,488 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=1f9ad9d3e00d7401d4d1052fa2e47826, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:21,488 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983261488"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983261488"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983261488"}]},"ts":"1675983261488"} 2023-02-09 22:54:21,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 1f9ad9d3e00d7401d4d1052fa2e47826, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:21,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:21,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1f9ad9d3e00d7401d4d1052fa2e47826, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:21,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:21,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,646 INFO [StoreOpener-1f9ad9d3e00d7401d4d1052fa2e47826-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,648 DEBUG [StoreOpener-1f9ad9d3e00d7401d4d1052fa2e47826-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:21,648 DEBUG [StoreOpener-1f9ad9d3e00d7401d4d1052fa2e47826-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:21,648 INFO [StoreOpener-1f9ad9d3e00d7401d4d1052fa2e47826-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f9ad9d3e00d7401d4d1052fa2e47826 columnFamilyName family1 2023-02-09 22:54:21,649 INFO [StoreOpener-1f9ad9d3e00d7401d4d1052fa2e47826-1] regionserver.HStore(310): Store=1f9ad9d3e00d7401d4d1052fa2e47826/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:21,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:54:21,655 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:21,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 1f9ad9d3e00d7401d4d1052fa2e47826; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71898496, jitterRate=0.07137107849121094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:21,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 1f9ad9d3e00d7401d4d1052fa2e47826: Running coprocessor pre-open hook at 1675983261645Writing region info on filesystem at 1675983261645Initializing all the Stores at 1675983261646 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983261646Cleaning up temporary data from old regions at 1675983261651 (+5 ms)Cleaning up detritus from prior splits at 1675983261651Running coprocessor post-open hooks at 1675983261656 (+5 ms)Region opened successfully at 1675983261656 2023-02-09 22:54:21,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826., pid=37, masterSystemTime=1675983261642 2023-02-09 22:54:21,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:21,659 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=1f9ad9d3e00d7401d4d1052fa2e47826, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:21,659 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675983261659"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983261659"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983261659"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983261659"}]},"ts":"1675983261659"} 2023-02-09 22:54:21,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-09 22:54:21,663 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 1f9ad9d3e00d7401d4d1052fa2e47826, server=jenkins-hbase9.apache.org,39093,1675983215026 in 171 msec 2023-02-09 22:54:21,664 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-09 22:54:21,664 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=1f9ad9d3e00d7401d4d1052fa2e47826, ASSIGN in 328 msec 2023-02-09 22:54:21,665 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:21,665 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983261665"}]},"ts":"1675983261665"} 2023-02-09 22:54:21,666 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-09 22:54:21,669 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:21,670 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1650 sec 2023-02-09 22:54:22,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 22:54:22,613 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-09 22:54:22,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:22,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,638 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:22,638 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:22,638 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:22,638 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:22,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,640 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:22,640 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:22,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:22,640 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:22,640 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:22,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:22,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:22,642 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:22,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:22,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:22,645 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:22,645 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:22,646 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:54:22,935 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:23,073 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a2bf7f88312847ed87f9117969dbe929 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2bf7f88312847ed87f9117969dbe929 2023-02-09 22:54:23,477 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/2cadd6f4a72c4bd3b4d42b89536ffeff 2023-02-09 22:54:23,482 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/2cadd6f4a72c4bd3b4d42b89536ffeff as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff 2023-02-09 22:54:23,486 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-09 22:54:23,487 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 842ms, sequenceid=5, compaction requested=false 2023-02-09 22:54:23,487 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:23,488 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:23,488 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:23,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:23,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:23,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:23,488 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:23,489 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,489 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:23,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:23,490 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:23,490 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:23,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:23,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:23,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:23,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c68d859[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:23,491 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:23,491 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,492 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,492 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:23,492 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:23,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,492 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:23,492 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:23,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:23,494 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:23,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:23,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:23,494 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:23,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:23,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:23,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:23,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:23,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:23,496 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:23,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:23,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:23,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,496 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:23,498 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,498 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:23,498 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:23,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:23,498 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,499 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:23,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:23,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:23,501 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:23,501 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:23,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,502 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:23,502 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:23,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:23,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:23,602 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:23,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:23,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:23,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:23,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,608 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:23,608 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:23,608 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:23,608 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:23,609 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,609 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,610 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,610 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:23,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:23,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:23,610 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:23,610 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:23,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:23,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:23,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:23,611 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:23,611 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:23,611 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:23,612 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:23,612 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:23,612 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:23,612 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:23,612 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:54:24,024 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b3d348b345674b77a5b5fd6a4ec8eb98 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128b3d348b345674b77a5b5fd6a4ec8eb98 2023-02-09 22:54:24,430 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/7b4e87bc4550465cbaca28d77e3d7b51 2023-02-09 22:54:24,435 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/7b4e87bc4550465cbaca28d77e3d7b51 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51 2023-02-09 22:54:24,438 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:54:24,439 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 827ms, sequenceid=9, compaction requested=false 2023-02-09 22:54:24,439 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:24,439 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:24,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:24,439 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:24,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:24,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:24,442 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,442 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:24,442 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,442 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:24,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:24,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b9c8dce[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:24,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:24,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,447 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,447 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:24,447 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:24,447 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:24,447 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,447 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:24,448 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:24,448 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,448 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:24,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,448 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:24,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:24,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:24,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:24,451 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:24,451 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:24,451 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:24,451 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,451 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:24,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:24,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:24,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:24,452 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,453 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:24,460 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:24,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:24,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:24,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:24,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:24,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,461 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,460 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:24,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:24,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:24,461 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:24,461 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:24,561 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:24,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:24,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:24,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:24,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,568 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:24,568 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:24,568 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:24,568 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:24,568 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,568 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,569 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:24,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:24,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:24,570 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:24,570 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:24,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:24,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:24,571 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:24,571 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:24,571 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:24,571 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:24,571 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:24,571 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:24,571 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:24,571 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:24,984 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011c0b5cb4974745c08b8dad539b9bbf45 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011c0b5cb4974745c08b8dad539b9bbf45 2023-02-09 22:54:25,389 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/97fa4904ebb64320b90772f91eb4e49b 2023-02-09 22:54:25,394 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/97fa4904ebb64320b90772f91eb4e49b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b 2023-02-09 22:54:25,398 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 22:54:25,399 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 828ms, sequenceid=13, compaction requested=true 2023-02-09 22:54:25,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:25,399 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:25,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:25,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:25,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:25,399 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:25,401 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,401 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:25,401 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,401 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69819a20[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:25,402 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:25,402 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,403 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,403 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:25,403 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:25,403 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,403 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:25,403 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:25,406 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,406 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:25,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:25,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:25,406 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:25,406 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:25,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:25,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:25,408 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:25,408 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:25,408 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:25,408 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,408 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:25,409 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,409 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:25,409 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:25,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:25,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:25,410 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,411 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:25,413 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:25,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:25,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:25,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:25,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,413 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:25,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:25,414 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:25,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:25,514 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:25,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:25,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:25,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:25,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,520 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:25,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:25,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:25,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:25,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,522 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,522 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:25,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:25,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:25,522 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:25,522 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:25,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:25,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:25,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:25,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:25,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:25,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:25,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:25,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:25,523 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:25,523 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:25,523 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:25,939 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c1bb899685ab48979cc5c3e299843132 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c1bb899685ab48979cc5c3e299843132 2023-02-09 22:54:26,351 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/e3c774eba4244231b6f16f273ab86911 2023-02-09 22:54:26,356 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/e3c774eba4244231b6f16f273ab86911 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911 2023-02-09 22:54:26,360 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 22:54:26,361 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 838ms, sequenceid=17, compaction requested=true 2023-02-09 22:54:26,361 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:26,361 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:26,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:26,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:26,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:26,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:26,364 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,364 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,365 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,365 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:26,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@326eb9a4[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:26,366 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:26,366 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,368 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:26,368 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:26,368 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:26,368 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,368 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:26,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:26,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,369 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,369 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:26,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:26,371 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:26,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:26,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:26,372 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,372 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:26,373 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,373 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:26,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,373 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:26,374 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,375 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:26,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:26,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:26,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,378 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,379 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,379 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:26,379 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:26,379 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:26,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:26,379 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:26,479 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:26,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:26,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:26,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:26,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,486 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:26,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:26,486 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:26,487 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,487 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,488 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:26,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:26,488 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:26,489 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:26,489 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:26,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:26,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:26,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:26,490 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:26,490 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:26,490 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:26,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:26,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:26,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:26,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:26,515 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:26,931 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228257650dd1b814784b78af850cba1d18c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228257650dd1b814784b78af850cba1d18c 2023-02-09 22:54:27,337 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/c43ea592691f40c388cb5b877e6049b4 2023-02-09 22:54:27,342 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/c43ea592691f40c388cb5b877e6049b4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4 2023-02-09 22:54:27,346 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 22:54:27,347 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 832ms, sequenceid=21, compaction requested=true 2023-02-09 22:54:27,347 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:27,347 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:27,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,348 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:27,348 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:27,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,357 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,357 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,357 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:27,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58ae05a0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,359 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:27,359 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,360 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:27,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:27,361 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:27,361 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,361 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,363 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,363 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,363 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,363 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:27,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:27,376 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,376 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:27,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,376 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,376 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,385 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,385 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,387 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,393 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,393 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,393 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:27,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,393 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,393 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,393 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,394 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,494 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:27,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:27,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:27,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:27,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,500 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:27,500 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,500 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:27,500 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:27,501 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,501 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,502 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,502 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,503 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,503 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:27,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,503 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:27,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,504 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:27,504 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:27,504 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:27,504 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:27,504 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:27,504 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:27,504 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:27,504 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:27,917 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ffe4e77e929e40439f179c70ed450a57 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ffe4e77e929e40439f179c70ed450a57 2023-02-09 22:54:27,922 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/b945987dee3549e5b23fa6f79e45b3d9 2023-02-09 22:54:27,929 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/b945987dee3549e5b23fa6f79e45b3d9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9 2023-02-09 22:54:27,934 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 22:54:27,935 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 431ms, sequenceid=25, compaction requested=true 2023-02-09 22:54:27,935 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:27,935 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:27,935 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:27,935 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:27,935 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,935 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:27,936 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:27,938 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,938 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,938 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,938 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:27,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:27,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67630186[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:27,940 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:27,940 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,941 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,941 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:27,941 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:27,941 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:27,941 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,941 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:27,943 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:27,943 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,943 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,943 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:27,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:27,946 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:27,946 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:27,946 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:27,946 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,946 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:27,947 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,947 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:27,947 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,948 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:27,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:27,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,949 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:27,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:27,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:27,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:27,953 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:27,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:27,953 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:27,954 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,954 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:27,954 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:27,954 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:27,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:27,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,054 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:28,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:28,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:28,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:28,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,060 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:28,061 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:28,061 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:28,061 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:28,061 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,061 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,062 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:28,062 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:28,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:28,063 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,063 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:28,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:28,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,064 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:28,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,064 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:28,064 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:28,064 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:28,064 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:28,064 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:28,064 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:28,064 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:28,064 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:28,476 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c670792201bb4ea6a37ebbf7ad8d4762 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c670792201bb4ea6a37ebbf7ad8d4762 2023-02-09 22:54:28,882 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/ea27df5277f842b09cbb4bf95b0db49c 2023-02-09 22:54:28,888 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/ea27df5277f842b09cbb4bf95b0db49c as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c 2023-02-09 22:54:28,893 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 22:54:28,894 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 830ms, sequenceid=29, compaction requested=true 2023-02-09 22:54:28,894 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:28,894 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:28,894 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:28,894 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:28,894 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,894 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:28,894 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:28,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,896 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:28,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:28,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d6c7d0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:28,898 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:28,898 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,899 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,899 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,899 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:28,899 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:28,899 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:28,899 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:28,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:28,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,905 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:28,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:28,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:28,908 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:28,908 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:28,908 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:28,908 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,908 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:28,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:28,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:28,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:28,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:28,910 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,911 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:28,916 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:28,916 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:28,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:28,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:28,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:28,916 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:28,917 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:28,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:28,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:29,017 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:29,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:29,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:29,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:29,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,023 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:29,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:29,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:29,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:29,024 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,024 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,026 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,026 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:29,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:29,026 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,026 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,027 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:29,027 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:29,027 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:29,027 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:29,028 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:29,028 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:29,028 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:29,028 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:29,442 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b0a9db919ef14dd6bd678ec4035ecc81 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0a9db919ef14dd6bd678ec4035ecc81 2023-02-09 22:54:29,449 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/e38217db8aa84efeac8ca3856946ed08 2023-02-09 22:54:29,455 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/e38217db8aa84efeac8ca3856946ed08 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08 2023-02-09 22:54:29,460 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 22:54:29,461 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 433ms, sequenceid=33, compaction requested=true 2023-02-09 22:54:29,462 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:29,462 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:29,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:29,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:29,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:29,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:29,464 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4532edb7[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:29,466 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:29,466 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,467 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(162): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:29,468 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:29,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:29,469 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:29,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:29,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:29,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:29,470 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:29,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:29,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:29,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:29,472 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:29,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:29,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:29,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,472 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:29,475 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,475 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,475 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:29,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:29,476 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:29,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:29,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,477 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,483 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:29,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:29,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:29,483 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:29,484 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:29,484 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:29,484 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,484 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:29,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:29,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:29,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:29,584 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:29,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:29,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:29,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:29,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,590 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:29,590 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:29,590 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:29,590 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:29,591 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,591 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,592 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:29,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:29,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:29,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:29,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:29,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:29,593 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:29,593 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:29,593 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:29,593 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:29,593 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:29,593 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:29,594 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:29,594 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:30,010 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a95218af9a0b47bdb2d98cea311a0a39 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907a95218af9a0b47bdb2d98cea311a0a39 2023-02-09 22:54:30,415 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/96b37b2413414e73875931a51c515b77 2023-02-09 22:54:30,420 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/96b37b2413414e73875931a51c515b77 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77 2023-02-09 22:54:30,424 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 22:54:30,425 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 831ms, sequenceid=37, compaction requested=true 2023-02-09 22:54:30,425 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:30,425 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:30,425 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:30,425 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:30,425 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,425 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:30,425 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:30,426 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,426 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:30,427 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,427 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:30,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:30,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d9973b4[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:30,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:30,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,429 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:30,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:30,429 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,429 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:30,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:30,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:30,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:30,430 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:30,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:30,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:30,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:30,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:30,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:30,432 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:30,432 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:30,432 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:30,432 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,433 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:30,435 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,435 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:30,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,435 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:30,436 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:30,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,437 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:30,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:30,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:30,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,439 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:30,440 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:30,439 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:30,440 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:30,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:30,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:30,540 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:30,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:30,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:30,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:30,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,546 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:30,546 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:30,546 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-09 22:54:30,546 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-09 22:54:30,546 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,546 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,548 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:30,548 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:30,548 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:30,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,548 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:30,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:30,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-09 22:54:30,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:30,549 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-09 22:54:30,549 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:30,549 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-09 22:54:30,549 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:30,549 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:30,549 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:30,549 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. started... 2023-02-09 22:54:30,549 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f9ad9d3e00d7401d4d1052fa2e47826 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:30,961 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a5dfe3f5906445d1911e8e2d0d62a590 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120a5dfe3f5906445d1911e8e2d0d62a590 2023-02-09 22:54:31,367 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/3a65c9e594674692861979e024d25557 2023-02-09 22:54:31,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/3a65c9e594674692861979e024d25557 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557 2023-02-09 22:54:31,375 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 22:54:31,376 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1f9ad9d3e00d7401d4d1052fa2e47826 in 827ms, sequenceid=41, compaction requested=true 2023-02-09 22:54:31,376 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:31,376 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:31,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:31,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:31,376 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,377 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-09 22:54:31,377 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-09 22:54:31,378 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,378 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:31,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:31,378 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,378 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:31,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:31,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:31,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:31,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-09 22:54:31,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@128930b0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:31,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-09 22:54:31,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,384 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,385 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-09 22:54:31,385 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-09 22:54:31,385 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,385 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:31,385 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:31,386 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-09 22:54:31,386 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,386 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:31,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:31,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:31,386 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-09 22:54:31,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:31,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:54:31,388 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-09 22:54:31,388 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:31,388 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:31,388 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,388 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:31,389 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,389 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:31,389 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:31,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:31,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:31,390 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,391 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:31,396 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:31,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:31,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-09 22:54:31,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:31,396 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:31,496 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:31,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:31,498 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-09 22:54:31,500 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:31,501 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:31,502 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:31,502 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983871502 2023-02-09 22:54:31,505 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:31,506 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-09 22:54:31,506 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:31,506 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x6404f1ea to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:31,510 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@755083bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:31,510 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,346 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010360075d8dfc8b4b4bb83a58a339d7d340 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010360075d8dfc8b4b4bb83a58a339d7d340 2023-02-09 22:54:32,347 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:32,360 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:32,361 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:57122, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:32,367 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:32,368 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:46454, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:32,376 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/64fcb40c6768418394438c365ad8f952 first=Optional[r3] last=Optional[r6] 2023-02-09 22:54:32,376 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/64fcb40c6768418394438c365ad8f952}] 2023-02-09 22:54:32,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/64fcb40c6768418394438c365ad8f952 for inclusion in 1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:32,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 22:54:32,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:32,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:32,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/64fcb40c6768418394438c365ad8f952 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r5ej7etkbansm6cf06vkhihbihak1dake1n3eudc09o8i80tp7sc60timdah8ece/family1/64fcb40c6768418394438c365ad8f952 2023-02-09 22:54:32,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r5ej7etkbansm6cf06vkhihbihak1dake1n3eudc09o8i80tp7sc60timdah8ece/family1/64fcb40c6768418394438c365ad8f952 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_ 2023-02-09 22:54:32,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r5ej7etkbansm6cf06vkhihbihak1dake1n3eudc09o8i80tp7sc60timdah8ece/family1/64fcb40c6768418394438c365ad8f952 into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_ - updating store file list. 2023-02-09 22:54:32,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_ into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:32,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r5ej7etkbansm6cf06vkhihbihak1dake1n3eudc09o8i80tp7sc60timdah8ece/family1/64fcb40c6768418394438c365ad8f952 into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_) 2023-02-09 22:54:32,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r5ej7etkbansm6cf06vkhihbihak1dake1n3eudc09o8i80tp7sc60timdah8ece/family1/64fcb40c6768418394438c365ad8f952 2023-02-09 22:54:32,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:54:32,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:54:32,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): 1f9ad9d3e00d7401d4d1052fa2e47826/family1 is initiating minor compaction 2023-02-09 22:54:32,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@31f5d34a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:32,393 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f9ad9d3e00d7401d4d1052fa2e47826/family1 in testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:32,393 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp, totalSize=50.0 K 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 7b4e87bc4550465cbaca28d77e3d7b51, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 97fa4904ebb64320b90772f91eb4e49b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting e3c774eba4244231b6f16f273ab86911, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c43ea592691f40c388cb5b877e6049b4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b945987dee3549e5b23fa6f79e45b3d9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting ea27df5277f842b09cbb4bf95b0db49c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting e38217db8aa84efeac8ca3856946ed08, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 96b37b2413414e73875931a51c515b77, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 3a65c9e594674692861979e024d25557, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:54:32,393 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:54:32,395 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:32,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:32,397 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228257650dd1b814784b78af850cba1d18c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228257650dd1b814784b78af850cba1d18c 2023-02-09 22:54:32,398 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ffe4e77e929e40439f179c70ed450a57 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231ffe4e77e929e40439f179c70ed450a57 2023-02-09 22:54:32,399 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c670792201bb4ea6a37ebbf7ad8d4762 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c670792201bb4ea6a37ebbf7ad8d4762 2023-02-09 22:54:32,400 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0a9db919ef14dd6bd678ec4035ecc81 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0a9db919ef14dd6bd678ec4035ecc81 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-09 22:54:32,401 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,401 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:32,427 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1f9ad9d3e00d7401d4d1052fa2e47826#family1#compaction#49 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:33,251 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/68745fcbdad7471cabf784a5b572f0e4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/68745fcbdad7471cabf784a5b572f0e4 2023-02-09 22:54:33,256 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1f9ad9d3e00d7401d4d1052fa2e47826/family1 of 1f9ad9d3e00d7401d4d1052fa2e47826 into 68745fcbdad7471cabf784a5b572f0e4(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:33,257 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:33,257 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675983272392; duration=0sec 2023-02-09 22:54:33,257 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:33,262 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205335046c3fd5244329b212e7c49a15baa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205335046c3fd5244329b212e7c49a15baa 2023-02-09 22:54:33,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:33,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_] to archive 2023-02-09 22:54:33,285 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:33,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/7b4e87bc4550465cbaca28d77e3d7b51 2023-02-09 22:54:33,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/97fa4904ebb64320b90772f91eb4e49b 2023-02-09 22:54:33,290 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e3c774eba4244231b6f16f273ab86911 2023-02-09 22:54:33,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c43ea592691f40c388cb5b877e6049b4 2023-02-09 22:54:33,293 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/b945987dee3549e5b23fa6f79e45b3d9 2023-02-09 22:54:33,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/ea27df5277f842b09cbb4bf95b0db49c 2023-02-09 22:54:33,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/e38217db8aa84efeac8ca3856946ed08 2023-02-09 22:54:33,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/96b37b2413414e73875931a51c515b77 2023-02-09 22:54:33,299 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/3a65c9e594674692861979e024d25557 2023-02-09 22:54:33,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c63a1e3c3068429e94d02d705b8c36b3_SeqId_44_ 2023-02-09 22:54:33,302 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/65a7fb96adf447c49a9e69394fe3c9fd first=Optional[r00] last=Optional[r2] 2023-02-09 22:54:33,303 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/65a7fb96adf447c49a9e69394fe3c9fd}] 2023-02-09 22:54:33,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/65a7fb96adf447c49a9e69394fe3c9fd for inclusion in 1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:33,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 22:54:33,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:33,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:33,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/65a7fb96adf447c49a9e69394fe3c9fd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vf15srkpdblpand7gbekqdgjsqato1ogjmb0ni0q9u5makeq48nv7oksi6nmr7kb/family1/65a7fb96adf447c49a9e69394fe3c9fd 2023-02-09 22:54:33,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vf15srkpdblpand7gbekqdgjsqato1ogjmb0ni0q9u5makeq48nv7oksi6nmr7kb/family1/65a7fb96adf447c49a9e69394fe3c9fd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_ 2023-02-09 22:54:33,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vf15srkpdblpand7gbekqdgjsqato1ogjmb0ni0q9u5makeq48nv7oksi6nmr7kb/family1/65a7fb96adf447c49a9e69394fe3c9fd into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_ - updating store file list. 2023-02-09 22:54:33,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_ into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 2023-02-09 22:54:33,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vf15srkpdblpand7gbekqdgjsqato1ogjmb0ni0q9u5makeq48nv7oksi6nmr7kb/family1/65a7fb96adf447c49a9e69394fe3c9fd into 1f9ad9d3e00d7401d4d1052fa2e47826/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_) 2023-02-09 22:54:33,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vf15srkpdblpand7gbekqdgjsqato1ogjmb0ni0q9u5makeq48nv7oksi6nmr7kb/family1/65a7fb96adf447c49a9e69394fe3c9fd 2023-02-09 22:54:33,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:54:33,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:54:33,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): 1f9ad9d3e00d7401d4d1052fa2e47826/family1 is initiating minor compaction (all files) 2023-02-09 22:54:33,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@174cdd1a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:33,317 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1f9ad9d3e00d7401d4d1052fa2e47826/family1 in testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:54:33,318 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/68745fcbdad7471cabf784a5b572f0e4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp, totalSize=337.0 K 2023-02-09 22:54:33,318 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2cadd6f4a72c4bd3b4d42b89536ffeff, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:54:33,318 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 68745fcbdad7471cabf784a5b572f0e4, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:54:33,319 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c6704e01561f47a0937be70d4fd0a23a_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839414 2023-02-09 22:54:33,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:33,320 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:33,321 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2bf7f88312847ed87f9117969dbe929 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2bf7f88312847ed87f9117969dbe929 2023-02-09 22:54:33,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011c0b5cb4974745c08b8dad539b9bbf45 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011c0b5cb4974745c08b8dad539b9bbf45 2023-02-09 22:54:33,324 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c1bb899685ab48979cc5c3e299843132 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c1bb899685ab48979cc5c3e299843132 2023-02-09 22:54:33,324 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-09 22:54:33,324 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 22:54:33,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-09 22:54:33,324 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-09 22:54:33,324 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:33,325 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:54:33,327 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8260 sec 2023-02-09 22:54:33,335 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1f9ad9d3e00d7401d4d1052fa2e47826#family1#compaction#50 average throughput is 22.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:33,413 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257 (was 249) - Thread LEAK? -, OpenFileDescriptor=616 (was 582) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 229), ProcessCount=168 (was 168), AvailableMemoryMB=10850 (was 10935) 2023-02-09 22:54:33,425 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=257, OpenFileDescriptor=616, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=168, AvailableMemoryMB=10850 2023-02-09 22:54:33,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:54:33,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:33,429 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:33,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-09 22:54:33,430 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:33,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:33,431 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:33,431 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d empty. 2023-02-09 22:54:33,432 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:33,432 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-09 22:54:33,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:33,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:33,840 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:33,841 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => f4484379e641d2df283701efc3eacc0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:34,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:34,246 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:34,247 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing f4484379e641d2df283701efc3eacc0d, disabling compactions & flushes 2023-02-09 22:54:34,247 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,247 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,247 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. after waiting 0 ms 2023-02-09 22:54:34,247 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,247 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,247 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for f4484379e641d2df283701efc3eacc0d: Waiting for close lock at 1675983274247Disabling compacts and flushes for region at 1675983274247Disabling writes for close at 1675983274247Writing region close event to WAL at 1675983274247Closed at 1675983274247 2023-02-09 22:54:34,249 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:34,250 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983274250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983274250"}]},"ts":"1675983274250"} 2023-02-09 22:54:34,251 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:34,252 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:34,252 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983274252"}]},"ts":"1675983274252"} 2023-02-09 22:54:34,253 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-09 22:54:34,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, ASSIGN}] 2023-02-09 22:54:34,258 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, ASSIGN 2023-02-09 22:54:34,259 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:34,410 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:34,410 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983274410"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983274410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983274410"}]},"ts":"1675983274410"} 2023-02-09 22:54:34,412 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:34,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:34,556 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/.tmp/family1/1e6cc64fef7d47b79b2454ec50bedf08 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/1e6cc64fef7d47b79b2454ec50bedf08 2023-02-09 22:54:34,561 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1f9ad9d3e00d7401d4d1052fa2e47826/family1 of 1f9ad9d3e00d7401d4d1052fa2e47826 into 1e6cc64fef7d47b79b2454ec50bedf08(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:54:34,561 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1f9ad9d3e00d7401d4d1052fa2e47826: 2023-02-09 22:54:34,561 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1675983273317; duration=1sec 2023-02-09 22:54:34,561 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:34,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f4484379e641d2df283701efc3eacc0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:34,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:34,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,568 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,569 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:34,569 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:34,569 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4484379e641d2df283701efc3eacc0d columnFamilyName family1 2023-02-09 22:54:34,571 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(310): Store=f4484379e641d2df283701efc3eacc0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:34,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:34,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:34,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f4484379e641d2df283701efc3eacc0d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71628485, jitterRate=0.06734760105609894}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:34,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f4484379e641d2df283701efc3eacc0d: Running coprocessor pre-open hook at 1675983274567Writing region info on filesystem at 1675983274567Initializing all the Stores at 1675983274568 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983274568Cleaning up temporary data from old regions at 1675983274572 (+4 ms)Cleaning up detritus from prior splits at 1675983274572Running coprocessor post-open hooks at 1675983274576 (+4 ms)Region opened successfully at 1675983274576 2023-02-09 22:54:34,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., pid=41, masterSystemTime=1675983274564 2023-02-09 22:54:34,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:34,579 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:34,579 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983274579"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983274579"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983274579"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983274579"}]},"ts":"1675983274579"} 2023-02-09 22:54:34,581 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-09 22:54:34,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 168 msec 2023-02-09 22:54:34,585 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/68745fcbdad7471cabf784a5b572f0e4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_] to archive 2023-02-09 22:54:34,586 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-09 22:54:34,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:34,586 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, ASSIGN in 324 msec 2023-02-09 22:54:34,586 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:34,587 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983274586"}]},"ts":"1675983274586"} 2023-02-09 22:54:34,588 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-09 22:54:34,588 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/2cadd6f4a72c4bd3b4d42b89536ffeff 2023-02-09 22:54:34,589 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/68745fcbdad7471cabf784a5b572f0e4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/68745fcbdad7471cabf784a5b572f0e4 2023-02-09 22:54:34,590 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:34,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/family1/c6704e01561f47a0937be70d4fd0a23a_SeqId_47_ 2023-02-09 22:54:34,591 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1630 sec 2023-02-09 22:54:35,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-09 22:54:35,535 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-09 22:54:35,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:35,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,557 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:35,557 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:35,557 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:35,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:35,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,559 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:35,559 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:35,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:35,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:35,559 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:35,559 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:35,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,560 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,560 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,560 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:35,560 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:35,560 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:35,560 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:35,561 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:35,561 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:35,561 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:35,561 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:35,561 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:54:35,979 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cf1768943a6d4546961bc94881a43521 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130cf1768943a6d4546961bc94881a43521 2023-02-09 22:54:36,383 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/35042b779e32492ea7adf33de07e2a5f 2023-02-09 22:54:36,388 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/35042b779e32492ea7adf33de07e2a5f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f 2023-02-09 22:54:36,392 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-09 22:54:36,392 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 831ms, sequenceid=5, compaction requested=false 2023-02-09 22:54:36,392 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:36,393 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:36,393 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:36,393 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:36,393 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:36,393 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,393 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:36,393 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:36,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,395 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:36,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:36,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:36,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26558d76[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:36,397 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:36,397 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,398 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,398 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:36,398 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:36,398 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:36,398 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,398 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:36,400 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,400 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:36,400 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:36,400 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:36,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:36,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:36,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:36,402 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:36,402 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:36,402 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:36,402 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,402 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:36,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:36,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:36,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:36,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:36,404 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,406 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:36,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:36,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:36,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:36,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:36,408 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:36,408 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:36,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:36,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:36,508 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:36,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:36,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:36,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:36,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,515 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:36,515 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:36,515 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:36,515 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:36,515 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,515 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,516 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,516 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:36,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:36,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:36,517 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:36,517 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:36,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:36,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:36,518 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:36,518 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:36,518 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:36,518 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:36,518 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:36,518 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:36,518 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:36,518 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:54:36,932 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288a4191b1d5f145b8a595e7fc3bcb2c8c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511288a4191b1d5f145b8a595e7fc3bcb2c8c 2023-02-09 22:54:36,965 DEBUG [regionserver/jenkins-hbase9:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 22:54:37,338 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/80ff244e30824471923eaa61c4d70ed7 2023-02-09 22:54:37,343 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/80ff244e30824471923eaa61c4d70ed7 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7 2023-02-09 22:54:37,347 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:54:37,348 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 830ms, sequenceid=9, compaction requested=false 2023-02-09 22:54:37,349 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:37,349 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:37,349 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:37,349 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:37,349 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,349 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:37,349 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:37,350 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,351 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:37,351 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:37,351 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:37,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:37,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7458935e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:37,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:37,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,354 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,354 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:37,354 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:37,354 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:37,354 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,354 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:37,355 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,355 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:37,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:37,355 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:37,356 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:37,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:37,358 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:37,358 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:37,358 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:37,358 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,358 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:37,360 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,360 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,360 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:37,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:37,360 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,361 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:37,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:37,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:37,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:37,367 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:37,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:37,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:37,367 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:37,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:37,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:37,468 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:37,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:37,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:37,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:37,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,474 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:37,474 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:37,474 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:37,474 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:37,474 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,474 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,476 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:37,476 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:37,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:37,476 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:37,476 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:37,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:37,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:37,477 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:37,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:37,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:37,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:37,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:37,477 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:37,477 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:37,477 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:37,477 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:37,889 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014324e5fc9aff4a9aa4a54d9af0148621 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014324e5fc9aff4a9aa4a54d9af0148621 2023-02-09 22:54:38,294 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/cbe593848930452d8fe145df8bd087a5 2023-02-09 22:54:38,299 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/cbe593848930452d8fe145df8bd087a5 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5 2023-02-09 22:54:38,304 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 22:54:38,304 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 827ms, sequenceid=13, compaction requested=true 2023-02-09 22:54:38,304 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:38,304 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:38,304 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:38,305 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:38,305 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,305 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:38,305 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:38,306 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:38,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:38,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:38,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:38,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:38,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:38,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:38,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@527a5ab2[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:38,308 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:38,308 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,310 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:38,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:38,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:38,310 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,310 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:38,312 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:38,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:38,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:38,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:38,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:38,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:38,314 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:38,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:38,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:38,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,314 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:38,315 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,315 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:38,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,315 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:38,316 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:38,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,317 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:38,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,319 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:38,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:38,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:38,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:38,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:38,319 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:38,420 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:38,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:38,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:38,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:38,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,426 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:38,426 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:38,426 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:38,426 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:38,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,428 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:38,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:38,429 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:38,429 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:38,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:38,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:38,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:38,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:38,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:38,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:38,430 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:38,430 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:38,430 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:38,843 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120526f8a5c9d91b456c81980ac62d994c29 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120526f8a5c9d91b456c81980ac62d994c29 2023-02-09 22:54:39,248 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/34ce0dff6e1d49b58a987c2e2d307b4f 2023-02-09 22:54:39,253 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/34ce0dff6e1d49b58a987c2e2d307b4f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f 2023-02-09 22:54:39,257 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 22:54:39,257 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 827ms, sequenceid=17, compaction requested=true 2023-02-09 22:54:39,257 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:39,257 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:39,257 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:39,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:39,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:39,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:39,259 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:39,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:39,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:39,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:39,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:39,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9adfcfd[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:39,261 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:39,261 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,262 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:39,262 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:39,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:39,262 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:39,263 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,263 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:39,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:39,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:39,263 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:39,263 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:39,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:39,265 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:39,265 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:39,265 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:39,265 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,265 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:39,268 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,268 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,268 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:39,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:39,268 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:39,269 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:39,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:39,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,271 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:39,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:39,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:39,372 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:39,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:39,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:39,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:39,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,378 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:39,378 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:39,378 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:39,378 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:39,378 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,379 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,380 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:39,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:39,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:39,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:39,380 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:39,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:39,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,381 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,381 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:39,381 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:39,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:39,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:39,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:39,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:39,381 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:39,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:39,381 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:39,381 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:39,792 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ae4ec832c35b4056b1d7c16dad78c318 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228ae4ec832c35b4056b1d7c16dad78c318 2023-02-09 22:54:40,094 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:40,198 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/0af3a20831d1408ea2d0548b1ae7671c 2023-02-09 22:54:40,203 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/0af3a20831d1408ea2d0548b1ae7671c as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c 2023-02-09 22:54:40,208 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 22:54:40,208 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 827ms, sequenceid=21, compaction requested=true 2023-02-09 22:54:40,208 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:40,208 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:40,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:40,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:40,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:40,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:40,214 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,214 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:40,216 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:40,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@399fe83[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:40,216 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,217 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,217 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:40,217 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:40,217 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:40,217 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,217 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:40,219 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:40,219 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:40,219 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,219 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:40,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:40,221 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:40,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:40,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:40,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,221 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:40,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:40,223 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,224 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:40,227 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:40,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:40,228 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:40,228 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:40,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:40,328 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:40,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:40,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:40,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:40,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,334 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:40,334 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:40,334 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:40,334 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:40,334 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,334 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,336 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:40,336 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:40,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:40,336 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,336 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:40,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:40,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:40,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:40,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:40,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:40,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:40,337 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:40,337 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:40,337 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:40,337 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:40,749 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311ed9044741ae416f8f95f8fd94d3eebf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311ed9044741ae416f8f95f8fd94d3eebf 2023-02-09 22:54:40,756 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/5a0bee9b6df5422896aa754363b41282 2023-02-09 22:54:40,760 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/5a0bee9b6df5422896aa754363b41282 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282 2023-02-09 22:54:40,764 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 22:54:40,765 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 428ms, sequenceid=25, compaction requested=true 2023-02-09 22:54:40,765 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:40,765 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:40,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:40,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:40,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:40,765 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:40,766 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,766 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,767 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,767 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@336b5fcd[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:40,768 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:40,769 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,769 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:40,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:40,770 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:40,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:40,771 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,771 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:40,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,771 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:40,771 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:40,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:40,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:40,773 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:40,773 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:40,773 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:40,773 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,773 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:40,776 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,776 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:40,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,776 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:40,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:40,776 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,778 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:40,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:40,780 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:40,780 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:40,880 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:40,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:40,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:40,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:40,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,887 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:40,887 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:40,887 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:40,887 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:40,887 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,887 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,888 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,888 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:40,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:40,889 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:40,889 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,890 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:40,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:40,890 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:40,890 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:40,890 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:40,890 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:40,890 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:40,890 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:40,890 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:40,890 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:41,305 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101a5afec97f8ec4e1696b6a141af29609d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5afec97f8ec4e1696b6a141af29609d 2023-02-09 22:54:41,710 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/3b9f619add79404bbcf75bd72481978d 2023-02-09 22:54:41,715 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/3b9f619add79404bbcf75bd72481978d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d 2023-02-09 22:54:41,719 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 22:54:41,720 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 830ms, sequenceid=29, compaction requested=true 2023-02-09 22:54:41,720 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:41,720 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:41,720 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:41,720 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:41,720 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,720 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:41,720 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:41,723 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,723 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:41,723 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,723 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:41,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:41,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33c43cd8[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:41,724 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:41,724 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,725 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,725 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:41,725 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:41,725 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:41,725 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,725 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:41,726 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,726 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:41,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:41,727 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:41,727 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:41,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:41,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:41,728 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:41,729 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:41,729 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:41,729 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,729 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:41,736 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:41,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:41,736 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:41,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,737 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:41,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:41,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:41,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:41,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:41,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:41,739 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:41,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:41,740 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:41,739 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:41,740 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,740 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,840 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:41,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:41,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:41,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:41,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,846 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:41,846 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:41,846 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:41,846 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:41,846 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,846 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,847 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,847 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:41,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:41,848 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:41,848 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:41,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:41,849 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:41,849 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:41,849 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:41,849 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:41,849 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:41,849 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:41,849 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:41,849 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:42,261 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e3cc005cfdc146808195d291bd74bac6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e3cc005cfdc146808195d291bd74bac6 2023-02-09 22:54:42,667 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/2fba188a0dc9497b89c66c09879266f3 2023-02-09 22:54:42,672 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/2fba188a0dc9497b89c66c09879266f3 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3 2023-02-09 22:54:42,676 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 22:54:42,677 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 828ms, sequenceid=33, compaction requested=true 2023-02-09 22:54:42,677 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:42,677 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:42,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:42,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:42,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:42,677 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:42,678 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,678 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:42,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:42,679 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,679 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:42,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:42,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55e86449[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:42,680 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:42,680 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,682 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:42,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:42,682 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:42,682 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,682 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:42,683 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:42,683 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:42,683 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,683 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:42,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:42,685 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:42,685 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:42,685 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:42,685 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,685 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:42,687 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,687 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:42,687 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:42,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:42,687 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:42,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,688 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:42,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:42,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:42,690 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:42,691 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:42,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:42,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:42,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:42,691 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,791 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:42,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:42,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:42,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:42,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,797 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:42,798 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:42,798 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:42,798 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:42,798 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,798 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,800 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,800 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:42,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:42,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:42,800 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:42,800 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:42,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,801 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:42,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:42,801 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:42,801 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:42,801 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:42,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:42,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:42,802 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:42,802 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:42,802 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:43,213 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075819843071424934bed6f02ccd24f4d9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509075819843071424934bed6f02ccd24f4d9 2023-02-09 22:54:43,619 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/89762ee8a868443098eb16cafd185b71 2023-02-09 22:54:43,623 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/89762ee8a868443098eb16cafd185b71 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71 2023-02-09 22:54:43,628 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 22:54:43,629 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 827ms, sequenceid=37, compaction requested=true 2023-02-09 22:54:43,629 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:43,629 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:43,629 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:43,629 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:43,629 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,629 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:43,629 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:43,631 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,631 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:43,631 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,631 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:43,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:43,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1949e591[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:43,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:43,632 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,633 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:43,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:43,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:43,633 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,633 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:43,635 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:43,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:43,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:43,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:43,637 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:43,637 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:43,637 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:43,637 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,637 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:43,639 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,639 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:43,639 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:43,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:43,640 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,641 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:43,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-09 22:54:43,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:43,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:43,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:43,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:43,645 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:43,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:43,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:43,746 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:43,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:43,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:43,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:43,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,752 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:43,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:43,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-09 22:54:43,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-09 22:54:43,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,753 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:43,753 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:43,754 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:43,754 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-09 22:54:43,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:43,755 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-09 22:54:43,755 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:43,755 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-09 22:54:43,755 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:43,755 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:43,755 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:43,755 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. started... 2023-02-09 22:54:43,755 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f4484379e641d2df283701efc3eacc0d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:44,167 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120544d0efe427e44fcb5b9cee1aabc7681 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120544d0efe427e44fcb5b9cee1aabc7681 2023-02-09 22:54:44,573 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/537e12bbe43e47819e508ffd99f1b85c 2023-02-09 22:54:44,577 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/537e12bbe43e47819e508ffd99f1b85c as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c 2023-02-09 22:54:44,581 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 22:54:44,582 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f4484379e641d2df283701efc3eacc0d in 827ms, sequenceid=41, compaction requested=true 2023-02-09 22:54:44,582 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:44,582 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:44,582 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:44,582 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:44,582 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,582 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-09 22:54:44,582 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-09 22:54:44,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,585 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:44,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:44,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-09 22:54:44,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70b55442[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:44,587 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-09 22:54:44,587 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,588 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-09 22:54:44,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-09 22:54:44,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:44,588 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,588 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-09 22:54:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:44,589 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,589 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-09 22:54:44,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:44,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:44,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-09 22:54:44,591 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-09 22:54:44,591 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:44,591 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:44,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,592 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:44,593 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,593 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,593 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:44,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:44,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:44,593 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,594 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:44,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:44,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-09 22:54:44,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:44,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:44,597 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:44,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,598 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:44,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,598 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:44,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,698 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:44,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:44,700 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:44,702 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:44,703 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:44,703 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:44,703 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983884703 2023-02-09 22:54:44,706 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:44,707 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 22:54:44,707 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:44,707 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x48414b56 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:44,713 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d29b078, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:44,713 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,551 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f3d1dda18f3f4f02804b5b664311bff7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f3d1dda18f3f4f02804b5b664311bff7 2023-02-09 22:54:45,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:45,564 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:45,565 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:32882, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:45,571 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:45,573 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52172, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:45,581 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/668e07c2b5284fd794e366199efde337 first=Optional[r3] last=Optional[r6] 2023-02-09 22:54:45,582 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/668e07c2b5284fd794e366199efde337}] 2023-02-09 22:54:45,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/668e07c2b5284fd794e366199efde337 for inclusion in f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:45,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 22:54:45,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:45,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:45,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/668e07c2b5284fd794e366199efde337 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__obm58p2tvtb57k6d24j5qtl231kcjhp125a7qsja2suet6to2nn1q72fo220d8jh/family1/668e07c2b5284fd794e366199efde337 2023-02-09 22:54:45,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__obm58p2tvtb57k6d24j5qtl231kcjhp125a7qsja2suet6to2nn1q72fo220d8jh/family1/668e07c2b5284fd794e366199efde337 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_ 2023-02-09 22:54:45,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__obm58p2tvtb57k6d24j5qtl231kcjhp125a7qsja2suet6to2nn1q72fo220d8jh/family1/668e07c2b5284fd794e366199efde337 into f4484379e641d2df283701efc3eacc0d/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_ - updating store file list. 2023-02-09 22:54:45,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_ into f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:45,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__obm58p2tvtb57k6d24j5qtl231kcjhp125a7qsja2suet6to2nn1q72fo220d8jh/family1/668e07c2b5284fd794e366199efde337 into f4484379e641d2df283701efc3eacc0d/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_) 2023-02-09 22:54:45,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__obm58p2tvtb57k6d24j5qtl231kcjhp125a7qsja2suet6to2nn1q72fo220d8jh/family1/668e07c2b5284fd794e366199efde337 2023-02-09 22:54:45,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:54:45,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:54:45,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): f4484379e641d2df283701efc3eacc0d/family1 is initiating minor compaction 2023-02-09 22:54:45,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@b3e946f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:45,599 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f4484379e641d2df283701efc3eacc0d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:45,600 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp, totalSize=50.4 K 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 80ff244e30824471923eaa61c4d70ed7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting cbe593848930452d8fe145df8bd087a5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 34ce0dff6e1d49b58a987c2e2d307b4f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 0af3a20831d1408ea2d0548b1ae7671c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 5a0bee9b6df5422896aa754363b41282, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 3b9f619add79404bbcf75bd72481978d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 2fba188a0dc9497b89c66c09879266f3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 89762ee8a868443098eb16cafd185b71, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 537e12bbe43e47819e508ffd99f1b85c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:54:45,600 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 19e562fa23c04ce3a35508946bab5091_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:54:45,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:45,602 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:45,604 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228ae4ec832c35b4056b1d7c16dad78c318 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228ae4ec832c35b4056b1d7c16dad78c318 2023-02-09 22:54:45,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311ed9044741ae416f8f95f8fd94d3eebf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512311ed9044741ae416f8f95f8fd94d3eebf 2023-02-09 22:54:45,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5afec97f8ec4e1696b6a141af29609d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5afec97f8ec4e1696b6a141af29609d 2023-02-09 22:54:45,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e3cc005cfdc146808195d291bd74bac6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103e3cc005cfdc146808195d291bd74bac6 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-09 22:54:45,608 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:45,627 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): f4484379e641d2df283701efc3eacc0d#family1#compaction#61 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:46,442 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2 2023-02-09 22:54:46,447 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f4484379e641d2df283701efc3eacc0d/family1 of f4484379e641d2df283701efc3eacc0d into 55ef50d8f9994bdcb1dfcc1ebcd1e8a2(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:54:46,447 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:46,447 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675983285599; duration=0sec 2023-02-09 22:54:46,447 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:46,448 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052617416f3e184ca8833512dcd77d5db4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512052617416f3e184ca8833512dcd77d5db4 2023-02-09 22:54:46,449 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:46,473 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2dd0502e96c64238acd11c61f6f0ce78 first=Optional[r00] last=Optional[r2] 2023-02-09 22:54:46,474 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2dd0502e96c64238acd11c61f6f0ce78}] 2023-02-09 22:54:46,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_] to archive 2023-02-09 22:54:46,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2dd0502e96c64238acd11c61f6f0ce78 for inclusion in f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:46,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:46,487 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/80ff244e30824471923eaa61c4d70ed7 2023-02-09 22:54:46,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 22:54:46,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:46,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:46,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2dd0502e96c64238acd11c61f6f0ce78 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5msmc5knkedrrfb5tie0lsrt6hursru80n5846d1l8h0ap6lb9isnhqgdkfqcp7h/family1/2dd0502e96c64238acd11c61f6f0ce78 2023-02-09 22:54:46,489 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/cbe593848930452d8fe145df8bd087a5 2023-02-09 22:54:46,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/34ce0dff6e1d49b58a987c2e2d307b4f 2023-02-09 22:54:46,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5msmc5knkedrrfb5tie0lsrt6hursru80n5846d1l8h0ap6lb9isnhqgdkfqcp7h/family1/2dd0502e96c64238acd11c61f6f0ce78 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_ 2023-02-09 22:54:46,492 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0af3a20831d1408ea2d0548b1ae7671c 2023-02-09 22:54:46,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5msmc5knkedrrfb5tie0lsrt6hursru80n5846d1l8h0ap6lb9isnhqgdkfqcp7h/family1/2dd0502e96c64238acd11c61f6f0ce78 into f4484379e641d2df283701efc3eacc0d/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_ - updating store file list. 2023-02-09 22:54:46,493 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/5a0bee9b6df5422896aa754363b41282 2023-02-09 22:54:46,495 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/3b9f619add79404bbcf75bd72481978d 2023-02-09 22:54:46,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_ into f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:46,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5msmc5knkedrrfb5tie0lsrt6hursru80n5846d1l8h0ap6lb9isnhqgdkfqcp7h/family1/2dd0502e96c64238acd11c61f6f0ce78 into f4484379e641d2df283701efc3eacc0d/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_) 2023-02-09 22:54:46,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5msmc5knkedrrfb5tie0lsrt6hursru80n5846d1l8h0ap6lb9isnhqgdkfqcp7h/family1/2dd0502e96c64238acd11c61f6f0ce78 2023-02-09 22:54:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:54:46,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/2fba188a0dc9497b89c66c09879266f3 2023-02-09 22:54:46,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:54:46,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): f4484379e641d2df283701efc3eacc0d/family1 is initiating minor compaction (all files) 2023-02-09 22:54:46,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7eed1303; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:46,499 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f4484379e641d2df283701efc3eacc0d/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:46,499 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp, totalSize=390.1 K 2023-02-09 22:54:46,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/89762ee8a868443098eb16cafd185b71 2023-02-09 22:54:46,500 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 35042b779e32492ea7adf33de07e2a5f, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:54:46,501 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 55ef50d8f9994bdcb1dfcc1ebcd1e8a2, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:54:46,501 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting e4e2680e1a174b928c10de048df7af18_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839414 2023-02-09 22:54:46,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:46,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/537e12bbe43e47819e508ffd99f1b85c 2023-02-09 22:54:46,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:46,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/19e562fa23c04ce3a35508946bab5091_SeqId_44_ 2023-02-09 22:54:46,505 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130cf1768943a6d4546961bc94881a43521 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130cf1768943a6d4546961bc94881a43521 2023-02-09 22:54:46,507 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014324e5fc9aff4a9aa4a54d9af0148621 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014324e5fc9aff4a9aa4a54d9af0148621 2023-02-09 22:54:46,509 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120526f8a5c9d91b456c81980ac62d994c29 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120526f8a5c9d91b456c81980ac62d994c29 2023-02-09 22:54:46,509 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-09 22:54:46,509 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 22:54:46,510 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:46,510 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:46,510 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:46,515 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:46,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8140 sec 2023-02-09 22:54:46,537 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): f4484379e641d2df283701efc3eacc0d#family1#compaction#62 average throughput is 10.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:54:46,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 22:54:46,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:46,998 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 22:54:46,999 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:46,999 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 22:54:47,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 22:54:47,004 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE}] 2023-02-09 22:54:47,004 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE 2023-02-09 22:54:47,005 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:47,005 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983287005"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983287005"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983287005"}]},"ts":"1675983287005"} 2023-02-09 22:54:47,006 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:47,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:47,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f4484379e641d2df283701efc3eacc0d, disabling compactions & flushes 2023-02-09 22:54:47,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:47,756 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/.tmp/family1/757fea88f51d4dafa812f61355c14c8f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/757fea88f51d4dafa812f61355c14c8f 2023-02-09 22:54:47,761 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f4484379e641d2df283701efc3eacc0d/family1 of f4484379e641d2df283701efc3eacc0d into 757fea88f51d4dafa812f61355c14c8f(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:54:47,761 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:47,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. after waiting 0 ms 2023-02-09 22:54:47,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:47,761 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1675983286499; duration=1sec 2023-02-09 22:54:47,761 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:54:47,762 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_] to archive 2023-02-09 22:54:47,763 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:47,765 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/35042b779e32492ea7adf33de07e2a5f 2023-02-09 22:54:47,766 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/55ef50d8f9994bdcb1dfcc1ebcd1e8a2 2023-02-09 22:54:47,767 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/e4e2680e1a174b928c10de048df7af18_SeqId_47_ 2023-02-09 22:54:47,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 22:54:47,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:47,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f4484379e641d2df283701efc3eacc0d: Waiting for close lock at 1675983287158Running coprocessor pre-close hooks at 1675983287158Disabling compacts and flushes for region at 1675983287158Disabling writes for close at 1675983287761 (+603 ms)Writing region close event to WAL at 1675983287769 (+8 ms)Running coprocessor post-close hooks at 1675983287772 (+3 ms)Closed at 1675983287772 2023-02-09 22:54:47,772 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: f4484379e641d2df283701efc3eacc0d to self. 2023-02-09 22:54:47,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:47,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=CLOSED 2023-02-09 22:54:47,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983287774"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983287774"}]},"ts":"1675983287774"} 2023-02-09 22:54:47,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 22:54:47,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 769 msec 2023-02-09 22:54:47,777 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=true 2023-02-09 22:54:47,927 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:47,927 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983287927"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983287927"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983287927"}]},"ts":"1675983287927"} 2023-02-09 22:54:47,929 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:48,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:48,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f4484379e641d2df283701efc3eacc0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:48,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:48,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,085 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,086 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:48,086 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:48,086 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4484379e641d2df283701efc3eacc0d columnFamilyName family1 2023-02-09 22:54:48,091 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/757fea88f51d4dafa812f61355c14c8f 2023-02-09 22:54:48,091 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(310): Store=f4484379e641d2df283701efc3eacc0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:48,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:48,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f4484379e641d2df283701efc3eacc0d; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68133364, jitterRate=0.015266239643096924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:48,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f4484379e641d2df283701efc3eacc0d: Running coprocessor pre-open hook at 1675983288084Writing region info on filesystem at 1675983288084Initializing all the Stores at 1675983288085 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675983288085Cleaning up temporary data from old regions at 1675983288093 (+8 ms)Cleaning up detritus from prior splits at 1675983288094 (+1 ms)Running coprocessor post-open hooks at 1675983288097 (+3 ms)Region opened successfully at 1675983288097 2023-02-09 22:54:48,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., pid=47, masterSystemTime=1675983288081 2023-02-09 22:54:48,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:48,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:48,099 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:48,099 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983288099"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983288099"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983288099"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983288099"}]},"ts":"1675983288099"} 2023-02-09 22:54:48,102 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-09 22:54:48,102 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 172 msec 2023-02-09 22:54:48,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 22:54:48,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE in 1.0980 sec 2023-02-09 22:54:48,105 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-09 22:54:48,105 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1030 sec 2023-02-09 22:54:48,107 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5170 sec 2023-02-09 22:54:48,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-09 22:54:48,108 INFO [Listener at localhost/33819] compactions.TestMobCompactor(278): alter status finished 2023-02-09 22:54:48,109 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:48,111 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:48,111 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:48,112 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:48,112 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983888112 2023-02-09 22:54:48,113 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:48,113 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 22:54:48,114 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:48,114 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x3d92f8ff to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:48,118 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@d083545, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:48,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,119 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,945 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112808b6bc086ff14b75a243638a70d26730 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112808b6bc086ff14b75a243638a70d26730 2023-02-09 22:54:48,946 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:48,958 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:54:48,959 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:32890, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:54:48,965 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:54:48,967 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:52186, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:54:48,974 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3aef35bfc5444bf1b09447119cacc2f9 first=Optional[r05] last=Optional[r8] 2023-02-09 22:54:48,975 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3aef35bfc5444bf1b09447119cacc2f9}] 2023-02-09 22:54:48,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3aef35bfc5444bf1b09447119cacc2f9 for inclusion in f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:48,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 22:54:48,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:54:48,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for f4484379e641d2df283701efc3eacc0d: 2023-02-09 22:54:48,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3aef35bfc5444bf1b09447119cacc2f9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4n7v2v8uhpsek8bdp34oe6nc2n5gbvko0vhbrdcujnob817vejir7u65h2f4q2o0/family1/3aef35bfc5444bf1b09447119cacc2f9 2023-02-09 22:54:48,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4n7v2v8uhpsek8bdp34oe6nc2n5gbvko0vhbrdcujnob817vejir7u65h2f4q2o0/family1/3aef35bfc5444bf1b09447119cacc2f9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0fb65bd575014fb2a0c3c4b768d03ac4_SeqId_53_ 2023-02-09 22:54:48,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4n7v2v8uhpsek8bdp34oe6nc2n5gbvko0vhbrdcujnob817vejir7u65h2f4q2o0/family1/3aef35bfc5444bf1b09447119cacc2f9 into f4484379e641d2df283701efc3eacc0d/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0fb65bd575014fb2a0c3c4b768d03ac4_SeqId_53_ - updating store file list. 2023-02-09 22:54:48,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0fb65bd575014fb2a0c3c4b768d03ac4_SeqId_53_ into f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:48,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4n7v2v8uhpsek8bdp34oe6nc2n5gbvko0vhbrdcujnob817vejir7u65h2f4q2o0/family1/3aef35bfc5444bf1b09447119cacc2f9 into f4484379e641d2df283701efc3eacc0d/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0fb65bd575014fb2a0c3c4b768d03ac4_SeqId_53_) 2023-02-09 22:54:48,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__4n7v2v8uhpsek8bdp34oe6nc2n5gbvko0vhbrdcujnob817vejir7u65h2f4q2o0/family1/3aef35bfc5444bf1b09447119cacc2f9 2023-02-09 22:54:48,991 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:48,991 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:54:48,993 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120544d0efe427e44fcb5b9cee1aabc7681 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120544d0efe427e44fcb5b9cee1aabc7681 2023-02-09 22:54:48,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511288a4191b1d5f145b8a595e7fc3bcb2c8c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511288a4191b1d5f145b8a595e7fc3bcb2c8c 2023-02-09 22:54:48,994 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-09 22:54:48,994 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 22:54:48,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,994 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:48,994 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:48,994 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:48,995 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:48,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 885 msec 2023-02-09 22:54:49,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:54:49,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:49,467 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 22:54:49,468 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-09 22:54:49,468 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-09 22:54:49,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-09 22:54:49,474 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE}] 2023-02-09 22:54:49,474 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE 2023-02-09 22:54:49,475 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:49,475 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983289475"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983289475"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983289475"}]},"ts":"1675983289475"} 2023-02-09 22:54:49,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:49,628 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f4484379e641d2df283701efc3eacc0d, disabling compactions & flushes 2023-02-09 22:54:49,628 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. after waiting 0 ms 2023-02-09 22:54:49,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,634 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 22:54:49,635 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,635 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f4484379e641d2df283701efc3eacc0d: Waiting for close lock at 1675983289628Running coprocessor pre-close hooks at 1675983289628Disabling compacts and flushes for region at 1675983289628Disabling writes for close at 1675983289628Writing region close event to WAL at 1675983289631 (+3 ms)Running coprocessor post-close hooks at 1675983289635 (+4 ms)Closed at 1675983289635 2023-02-09 22:54:49,635 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: f4484379e641d2df283701efc3eacc0d to self. 2023-02-09 22:54:49,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,637 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=CLOSED 2023-02-09 22:54:49,637 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983289637"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983289637"}]},"ts":"1675983289637"} 2023-02-09 22:54:49,639 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-09 22:54:49,640 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 161 msec 2023-02-09 22:54:49,640 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=true 2023-02-09 22:54:49,790 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:49,791 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983289790"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983289790"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983289790"}]},"ts":"1675983289790"} 2023-02-09 22:54:49,792 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:49,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f4484379e641d2df283701efc3eacc0d, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:49,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:49,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,949 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,950 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:49,950 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1 2023-02-09 22:54:49,951 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f4484379e641d2df283701efc3eacc0d columnFamilyName family1 2023-02-09 22:54:49,955 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/0fb65bd575014fb2a0c3c4b768d03ac4_SeqId_53_ 2023-02-09 22:54:49,958 DEBUG [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/family1/757fea88f51d4dafa812f61355c14c8f 2023-02-09 22:54:49,958 INFO [StoreOpener-f4484379e641d2df283701efc3eacc0d-1] regionserver.HStore(310): Store=f4484379e641d2df283701efc3eacc0d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:49,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:54:49,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened f4484379e641d2df283701efc3eacc0d; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62468790, jitterRate=-0.06914249062538147}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:49,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for f4484379e641d2df283701efc3eacc0d: Running coprocessor pre-open hook at 1675983289948Writing region info on filesystem at 1675983289948Initializing all the Stores at 1675983289949 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983289949Cleaning up temporary data from old regions at 1675983289960 (+11 ms)Cleaning up detritus from prior splits at 1675983289961 (+1 ms)Running coprocessor post-open hooks at 1675983289963 (+2 ms)Region opened successfully at 1675983289963 2023-02-09 22:54:49,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d., pid=53, masterSystemTime=1675983289944 2023-02-09 22:54:49,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:54:49,966 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=f4484379e641d2df283701efc3eacc0d, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:49,966 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675983289965"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983289965"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983289965"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983289965"}]},"ts":"1675983289965"} 2023-02-09 22:54:49,969 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-09 22:54:49,969 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure f4484379e641d2df283701efc3eacc0d, server=jenkins-hbase9.apache.org,39093,1675983215026 in 175 msec 2023-02-09 22:54:49,970 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 22:54:49,970 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=f4484379e641d2df283701efc3eacc0d, REOPEN/MOVE in 495 msec 2023-02-09 22:54:49,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 22:54:49,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 501 msec 2023-02-09 22:54:49,974 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 917 msec 2023-02-09 22:54:49,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 22:54:49,976 INFO [Listener at localhost/33819] compactions.TestMobCompactor(278): alter status finished 2023-02-09 22:54:49,977 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-09 22:54:49,978 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:49,979 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:49,979 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:49,979 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983889979 2023-02-09 22:54:49,980 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:54:49,981 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-09 22:54:49,981 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:49,981 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x20f2900f to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:54:49,985 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c9c2b49, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-09 22:54:49,986 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-09 22:54:49,986 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:54:49,987 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-09 22:54:49,988 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-02-09 22:54:50,048 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=246 (was 257), OpenFileDescriptor=622 (was 616) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=146 (was 193), ProcessCount=168 (was 168), AvailableMemoryMB=10838 (was 10850) 2023-02-09 22:54:50,059 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=246, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=146, ProcessCount=168, AvailableMemoryMB=10837 2023-02-09 22:54:50,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-09 22:54:50,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:50,064 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:54:50,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-09 22:54:50,064 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:54:50,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:50,065 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:50,066 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 empty. 2023-02-09 22:54:50,066 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:50,066 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-09 22:54:50,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:50,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:50,475 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 22:54:50,476 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => e645e62e6f6942b9a9c124b648b04211, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:54:50,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:50,881 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:50,881 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing e645e62e6f6942b9a9c124b648b04211, disabling compactions & flushes 2023-02-09 22:54:50,881 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:50,881 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:50,881 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. after waiting 0 ms 2023-02-09 22:54:50,881 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:50,882 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:50,882 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for e645e62e6f6942b9a9c124b648b04211: Waiting for close lock at 1675983290881Disabling compacts and flushes for region at 1675983290881Disabling writes for close at 1675983290881Writing region close event to WAL at 1675983290882 (+1 ms)Closed at 1675983290882 2023-02-09 22:54:50,885 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:54:50,886 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983290886"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983290886"}]},"ts":"1675983290886"} 2023-02-09 22:54:50,887 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:54:50,888 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:54:50,888 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983290888"}]},"ts":"1675983290888"} 2023-02-09 22:54:50,889 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-09 22:54:50,891 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, ASSIGN}] 2023-02-09 22:54:50,892 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, ASSIGN 2023-02-09 22:54:50,893 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:54:51,044 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:51,044 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983291044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983291044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983291044"}]},"ts":"1675983291044"} 2023-02-09 22:54:51,045 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:54:51,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:51,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:51,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e645e62e6f6942b9a9c124b648b04211, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:54:51,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:54:51,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,201 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,202 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:54:51,202 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:54:51,203 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e645e62e6f6942b9a9c124b648b04211 columnFamilyName family1 2023-02-09 22:54:51,204 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(310): Store=e645e62e6f6942b9a9c124b648b04211/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:54:51,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:54:51,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:54:51,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened e645e62e6f6942b9a9c124b648b04211; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72039491, jitterRate=0.07347206771373749}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:54:51,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for e645e62e6f6942b9a9c124b648b04211: Running coprocessor pre-open hook at 1675983291200Writing region info on filesystem at 1675983291200Initializing all the Stores at 1675983291201 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675983291201Cleaning up temporary data from old regions at 1675983291205 (+4 ms)Cleaning up detritus from prior splits at 1675983291206 (+1 ms)Running coprocessor post-open hooks at 1675983291210 (+4 ms)Region opened successfully at 1675983291210 2023-02-09 22:54:51,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., pid=57, masterSystemTime=1675983291197 2023-02-09 22:54:51,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:51,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:51,212 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:51,212 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983291212"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983291212"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983291212"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983291212"}]},"ts":"1675983291212"} 2023-02-09 22:54:51,214 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 22:54:51,214 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026 in 168 msec 2023-02-09 22:54:51,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-09 22:54:51,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, ASSIGN in 323 msec 2023-02-09 22:54:51,216 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:54:51,216 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983291216"}]},"ts":"1675983291216"} 2023-02-09 22:54:51,217 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-09 22:54:51,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:54:51,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1580 sec 2023-02-09 22:54:52,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-09 22:54:52,170 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-09 22:54:52,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:52,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,191 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:52,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:52,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:52,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:52,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,191 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,194 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:52,194 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:52,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:52,194 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:52,194 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:52,195 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:52,195 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:52,196 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:52,196 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:52,196 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:52,196 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:52,196 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:52,197 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:54:52,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a6a96f09102a44258690468f2005de01 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a6a96f09102a44258690468f2005de01 2023-02-09 22:54:53,025 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/343c92148e5b412f85084fe02eb69c9b 2023-02-09 22:54:53,030 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/343c92148e5b412f85084fe02eb69c9b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b 2023-02-09 22:54:53,034 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-09 22:54:53,035 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 838ms, sequenceid=5, compaction requested=false 2023-02-09 22:54:53,035 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:53,035 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:53,035 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:53,035 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:53,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:53,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:53,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:53,037 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,037 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:53,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:53,037 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,037 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:53,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:53,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:53,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62859470[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:53,039 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:53,039 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,040 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,040 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:53,040 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:53,040 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:53,040 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,040 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:53,041 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:53,041 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:53,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,041 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:53,043 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:53,043 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:53,043 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:53,043 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,044 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:53,046 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,046 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,046 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:53,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:53,046 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,047 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,052 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:53,052 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:53,052 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:53,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 22:54:53,052 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:53,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:53,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:53,052 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:53,053 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:53,153 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:53,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:53,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:53,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:53,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,159 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:53,159 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:53,159 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:53,159 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:53,160 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,160 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,161 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,161 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:53,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:53,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:53,161 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,161 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:53,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:53,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,162 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:53,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,162 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:53,162 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:53,162 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:53,162 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:53,162 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:53,162 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:53,162 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:53,163 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:54:53,321 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:54:53,577 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112826a4b8457fb84b6d8b3157dfe5523132 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112826a4b8457fb84b6d8b3157dfe5523132 2023-02-09 22:54:53,982 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37 2023-02-09 22:54:53,987 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37 2023-02-09 22:54:53,991 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:54:53,991 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 829ms, sequenceid=9, compaction requested=false 2023-02-09 22:54:53,992 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:53,992 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:53,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:53,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:53,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:53,992 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:53,994 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:53,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:53,994 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:53,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:53,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:53,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:53,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74596e45[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:53,996 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:53,996 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,997 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,997 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:53,997 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:53,997 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:53,997 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,997 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:53,998 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,998 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:53,998 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:53,998 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:53,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:53,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:53,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:53,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:54,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:54,000 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:54,001 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:54,001 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:54,001 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,001 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:54,002 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,002 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:54,002 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:54,002 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,003 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:54,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:54,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:54,007 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,007 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,107 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:54,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:54,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:54,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:54,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,113 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:54,114 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:54,114 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:54,114 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:54,114 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,114 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,115 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:54,115 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:54,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:54,116 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,116 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:54,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:54,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,117 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:54,117 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,117 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:54,117 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:54,117 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:54,117 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:54,117 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:54,117 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:54,117 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:54,117 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:54,530 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011bcf174282054ab8b5b64f960479b6d1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512011bcf174282054ab8b5b64f960479b6d1 2023-02-09 22:54:54,935 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/5d26ef0e7f5444e8ba211fdd2860bc4d 2023-02-09 22:54:54,939 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/5d26ef0e7f5444e8ba211fdd2860bc4d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d 2023-02-09 22:54:54,943 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d, entries=1, sequenceid=13, filesize=5.0 K 2023-02-09 22:54:54,944 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 827ms, sequenceid=13, compaction requested=true 2023-02-09 22:54:54,944 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:54,944 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:54,944 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:54,944 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:54,944 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,944 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:54,944 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:54,946 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,946 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:54,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:54,946 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,946 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:54,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:54,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:54,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22ea7832[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:54,948 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:54,948 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,949 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:54,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:54,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:54,949 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,949 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:54,951 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,951 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:54,951 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:54,951 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:54,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:54,953 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:54,953 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:54,953 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:54,953 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,953 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:54,955 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,955 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:54,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,955 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:54,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:54,956 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:54,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,957 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:54,959 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:54,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:54,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:54,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:54,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:54,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:54,959 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:55,059 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:55,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:55,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:55,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:55,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,065 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:55,065 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:55,065 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:55,065 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:55,066 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,066 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,067 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,067 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:55,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:55,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:55,067 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,067 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:55,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:55,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,068 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:55,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,068 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:55,068 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:55,068 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:55,068 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:55,068 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:55,069 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:55,069 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:55,069 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:55,481 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058eabb98729144a4997ace49f8e1d35d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058eabb98729144a4997ace49f8e1d35d2 2023-02-09 22:54:55,887 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/501fc049e49943f99292135078139ab4 2023-02-09 22:54:55,891 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/501fc049e49943f99292135078139ab4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4 2023-02-09 22:54:55,895 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4, entries=1, sequenceid=17, filesize=5.0 K 2023-02-09 22:54:55,895 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 826ms, sequenceid=17, compaction requested=true 2023-02-09 22:54:55,895 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:55,895 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:55,895 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:55,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:55,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:55,896 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:55,898 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,898 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:55,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:55,898 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,898 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:55,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:55,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78a72f69[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:55,899 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:55,900 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,900 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,901 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:55,901 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:55,901 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,901 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:55,901 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:55,905 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:55,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:55,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:55,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:55,905 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:55,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:55,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:55,907 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:55,907 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:55,907 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:55,907 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,907 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:55,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:55,909 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,910 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:55,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,911 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,912 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,912 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,912 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:55,912 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:55,913 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:55,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:55,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:55,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:55,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:55,913 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:55,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:56,013 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:56,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:56,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:56,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:56,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,020 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:56,020 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:56,020 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:56,020 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:56,020 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,020 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,021 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,021 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:56,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:56,022 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,022 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:56,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:56,023 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,023 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:56,023 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:56,023 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:56,023 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:56,023 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:56,023 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:56,023 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:56,023 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:56,036 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228aa6f4af63300421c97ec6c2738a4b0ec to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228aa6f4af63300421c97ec6c2738a4b0ec 2023-02-09 22:54:56,446 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/52f81befbce0428d9462a039685b88a1 2023-02-09 22:54:56,452 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/52f81befbce0428d9462a039685b88a1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1 2023-02-09 22:54:56,457 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1, entries=1, sequenceid=21, filesize=5.0 K 2023-02-09 22:54:56,457 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 434ms, sequenceid=21, compaction requested=true 2023-02-09 22:54:56,457 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:56,458 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:56,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:56,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:56,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:56,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:56,459 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,459 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:56,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:56,460 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,460 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:56,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:56,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:56,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:56,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:56,461 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@364f1219[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:56,461 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:56,461 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,464 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,464 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:56,464 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:56,464 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:56,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:56,465 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:56,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:56,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:56,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:56,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:56,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:56,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:56,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:56,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:56,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:56,468 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:56,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:56,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:56,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,468 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:56,469 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,469 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:56,469 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,470 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:56,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,471 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:56,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:56,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:56,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,473 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:56,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,473 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:56,573 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:56,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:56,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:56,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:56,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,579 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:56,579 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:56,579 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:56,579 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:56,580 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,580 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,582 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:56,582 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:56,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:56,582 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:56,582 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:56,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:56,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,583 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:56,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:56,584 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:56,584 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:56,584 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:56,584 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:56,584 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:56,584 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:56,584 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:56,584 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:56,998 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123128d8d1ac6f1d45a3ac9ba2424f326b4f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123128d8d1ac6f1d45a3ac9ba2424f326b4f 2023-02-09 22:54:57,021 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:57,021 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6404f1ea to 127.0.0.1:59035 2023-02-09 22:54:57,021 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:57,023 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:57,023 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0085e7ca to 127.0.0.1:59035 2023-02-09 22:54:57,023 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:57,031 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:57,031 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x48414b56 to 127.0.0.1:59035 2023-02-09 22:54:57,031 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:57,031 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20f2900f to 127.0.0.1:59035 2023-02-09 22:54:57,031 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:57,031 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:54:57,031 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d92f8ff to 127.0.0.1:59035 2023-02-09 22:54:57,031 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:54:57,427 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/78dd8009acf44db0b55ab9a24df20f7e 2023-02-09 22:54:57,431 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/78dd8009acf44db0b55ab9a24df20f7e as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e 2023-02-09 22:54:57,435 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e, entries=1, sequenceid=25, filesize=5.0 K 2023-02-09 22:54:57,436 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 852ms, sequenceid=25, compaction requested=true 2023-02-09 22:54:57,436 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:57,436 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:57,436 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:57,436 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:57,436 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,436 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:57,436 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:57,437 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,437 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:57,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:57,438 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,438 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:57,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:57,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:57,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:57,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:57,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@95b69d3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:57,439 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:57,439 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,440 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:57,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:57,440 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:57,440 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,440 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:57,441 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:57,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:57,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:57,441 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:57,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:57,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:57,443 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:57,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:57,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:57,443 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,443 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:57,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:57,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,446 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:57,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:57,447 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:57,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,448 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:57,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,451 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,451 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:57,451 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:57,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:57,452 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:57,451 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:57,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:57,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:57,451 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:57,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:57,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:57,452 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,552 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:57,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:57,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:57,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:57,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,558 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:57,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:57,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:57,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:57,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,558 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,562 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:57,562 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:57,563 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:57,563 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:57,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:57,564 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:57,564 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:57,564 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:57,564 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:57,564 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:57,564 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:57,564 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:57,564 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:57,975 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019bd24789afb94b0dbfe98b5df5b54800 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019bd24789afb94b0dbfe98b5df5b54800 2023-02-09 22:54:58,380 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/1dc2b90bd6914228859c4d4bd9e1d30e 2023-02-09 22:54:58,385 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/1dc2b90bd6914228859c4d4bd9e1d30e as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e 2023-02-09 22:54:58,389 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e, entries=1, sequenceid=29, filesize=5.0 K 2023-02-09 22:54:58,389 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 825ms, sequenceid=29, compaction requested=true 2023-02-09 22:54:58,389 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:58,389 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:58,390 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:58,390 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:58,390 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,390 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:58,390 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:58,392 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,392 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:58,392 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,392 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:58,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:58,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:58,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7801bb22[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:58,393 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:58,393 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,394 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,394 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:58,394 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:58,394 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:58,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,394 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:58,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:58,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:58,395 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,395 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:58,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:58,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:58,398 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:58,398 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:58,398 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:58,398 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,398 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:58,399 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,399 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,399 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:58,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:58,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:58,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:58,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:58,400 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:58,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,401 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:58,404 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:58,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:58,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:58,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:58,405 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:58,404 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:58,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:58,405 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,505 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:58,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:58,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:58,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:58,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,511 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:58,511 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:58,511 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:58,511 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:58,512 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,512 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,513 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:58,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:58,513 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:58,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,514 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:58,514 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:58,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:58,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:58,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:58,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:58,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:58,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:58,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:58,515 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:58,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:58,515 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:58,515 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:58,928 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034168a377b4124c3091cefbe536a225ba to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034168a377b4124c3091cefbe536a225ba 2023-02-09 22:54:59,333 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/4f55e8220e294337a019bbab095e9aab 2023-02-09 22:54:59,337 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/4f55e8220e294337a019bbab095e9aab as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab 2023-02-09 22:54:59,341 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab, entries=1, sequenceid=33, filesize=5.0 K 2023-02-09 22:54:59,342 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 827ms, sequenceid=33, compaction requested=true 2023-02-09 22:54:59,342 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:54:59,342 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:59,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:54:59,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:54:59,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:54:59,342 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:54:59,344 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,344 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:59,344 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,344 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:59,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:59,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:54:59,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@399c1f2c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:54:59,346 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:54:59,346 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,347 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:54:59,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:54:59,347 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:54:59,347 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,347 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:54:59,350 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,350 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:54:59,350 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:59,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:59,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:59,350 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:54:59,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:59,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:59,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:59,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:54:59,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:54:59,352 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:54:59,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:54:59,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:54:59,352 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,352 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:54:59,354 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,354 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:54:59,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,354 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:54:59,354 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,355 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:54:59,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,357 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:54:59,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 22:54:59,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:59,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:59,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:54:59,357 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,358 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,358 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,358 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:54:59,358 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:54:59,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:54:59,358 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:54:59,458 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:54:59,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:54:59,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:54:59,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:54:59,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,464 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:54:59,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:59,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:54:59,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:54:59,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,468 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:54:59,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:54:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:54:59,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:54:59,468 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:54:59,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:54:59,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,469 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:54:59,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:54:59,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:54:59,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:54:59,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:54:59,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:54:59,469 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:54:59,469 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:54:59,469 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:54:59,470 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:54:59,887 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907232329e6e5004e5ca5f09ca25f0fc977 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907232329e6e5004e5ca5f09ca25f0fc977 2023-02-09 22:55:00,296 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/881ba73b214f42a08ab92901a21f0b07 2023-02-09 22:55:00,300 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/881ba73b214f42a08ab92901a21f0b07 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07 2023-02-09 22:55:00,305 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07, entries=1, sequenceid=37, filesize=5.0 K 2023-02-09 22:55:00,306 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 835ms, sequenceid=37, compaction requested=true 2023-02-09 22:55:00,306 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:00,306 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:00,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:00,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:00,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:55:00,306 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:55:00,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,308 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:00,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,308 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:00,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:00,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:55:00,309 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:55:00,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@778d0ddc[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:00,309 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,310 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:55:00,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:55:00,310 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:00,311 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,311 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:00,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:55:00,312 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:00,312 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:55:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:00,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:00,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:00,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:55:00,314 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:55:00,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:00,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:00,314 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,314 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:00,317 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,317 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,317 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:00,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:00,317 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,318 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,320 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:00,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:55:00,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:00,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:00,320 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,321 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,321 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:00,321 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:00,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-09 22:55:00,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:00,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:00,321 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,321 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,421 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:00,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:00,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:00,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:00,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,427 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:55:00,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:00,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-09 22:55:00,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-09 22:55:00,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,427 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,428 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:00,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:00,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:00,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:00,428 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-09 22:55:00,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:00,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-09 22:55:00,429 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:00,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-09 22:55:00,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:00,430 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:00,430 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:00,430 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. started... 2023-02-09 22:55:00,430 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e645e62e6f6942b9a9c124b648b04211 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:55:00,841 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112004144c5208e94700b3839d87b7d71510 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112004144c5208e94700b3839d87b7d71510 2023-02-09 22:55:01,246 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/385b49f1e6064fae98a2d7311a2e2eaa 2023-02-09 22:55:01,250 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/385b49f1e6064fae98a2d7311a2e2eaa as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa 2023-02-09 22:55:01,253 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa, entries=1, sequenceid=41, filesize=5.0 K 2023-02-09 22:55:01,254 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e645e62e6f6942b9a9c124b648b04211 in 824ms, sequenceid=41, compaction requested=true 2023-02-09 22:55:01,254 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:01,254 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-09 22:55:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-09 22:55:01,256 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,256 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:01,256 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,256 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:01,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-09 22:55:01,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25cae70f[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:01,257 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-09 22:55:01,257 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,258 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-09 22:55:01,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-09 22:55:01,258 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:01,258 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,258 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:01,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-09 22:55:01,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:01,259 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,259 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:01,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-09 22:55:01,261 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-09 22:55:01,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:01,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:01,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,262 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:01,264 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,264 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:01,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,264 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:01,264 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:01,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,265 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:01,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:01,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-09 22:55:01,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:01,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,267 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:01,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:01,267 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:01,268 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:01,368 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:01,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:01,369 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:01,371 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:01,372 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:01,372 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:01,372 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983901372 2023-02-09 22:55:01,375 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:55:01,375 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 22:55:01,375 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:01,376 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x14035645 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:01,380 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7842fcd8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:01,380 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,822 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103de0849579ff346f19fb95b9640c226e5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103de0849579ff346f19fb95b9640c226e5 2023-02-09 22:55:01,823 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:01,835 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:01,836 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33084, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:01,842 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:01,842 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49382, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:01,850 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/89739d07411a44599d6cdc47b3c27535 first=Optional[r3] last=Optional[r6] 2023-02-09 22:55:01,850 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/89739d07411a44599d6cdc47b3c27535}] 2023-02-09 22:55:01,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/89739d07411a44599d6cdc47b3c27535 for inclusion in e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:01,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-09 22:55:01,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:55:01,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:01,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/89739d07411a44599d6cdc47b3c27535 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r6rkopeduircbqonsv5odb044eo3mbf1f8vmt25vpn417rf5q2on0t933hcnu8pr/family1/89739d07411a44599d6cdc47b3c27535 2023-02-09 22:55:01,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r6rkopeduircbqonsv5odb044eo3mbf1f8vmt25vpn417rf5q2on0t933hcnu8pr/family1/89739d07411a44599d6cdc47b3c27535 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_ 2023-02-09 22:55:01,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r6rkopeduircbqonsv5odb044eo3mbf1f8vmt25vpn417rf5q2on0t933hcnu8pr/family1/89739d07411a44599d6cdc47b3c27535 into e645e62e6f6942b9a9c124b648b04211/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_ - updating store file list. 2023-02-09 22:55:01,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_ into e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:01,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r6rkopeduircbqonsv5odb044eo3mbf1f8vmt25vpn417rf5q2on0t933hcnu8pr/family1/89739d07411a44599d6cdc47b3c27535 into e645e62e6f6942b9a9c124b648b04211/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_) 2023-02-09 22:55:01,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__r6rkopeduircbqonsv5odb044eo3mbf1f8vmt25vpn417rf5q2on0t933hcnu8pr/family1/89739d07411a44599d6cdc47b3c27535 2023-02-09 22:55:01,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:55:01,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:55:01,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): e645e62e6f6942b9a9c124b648b04211/family1 is initiating minor compaction 2023-02-09 22:55:01,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5f906d74; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:01,865 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e645e62e6f6942b9a9c124b648b04211/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:01,865 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp, totalSize=50.3 K 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 3d56ed8ffe8a40f3a2e0dc2de147fe37, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 5d26ef0e7f5444e8ba211fdd2860bc4d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 501fc049e49943f99292135078139ab4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 52f81befbce0428d9462a039685b88a1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 78dd8009acf44db0b55ab9a24df20f7e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 1dc2b90bd6914228859c4d4bd9e1d30e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 4f55e8220e294337a019bbab095e9aab, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 881ba73b214f42a08ab92901a21f0b07, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 385b49f1e6064fae98a2d7311a2e2eaa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:55:01,866 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c47fce6cdb754262a195362978d62df5_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:55:01,867 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:01,868 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:01,869 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228aa6f4af63300421c97ec6c2738a4b0ec to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228aa6f4af63300421c97ec6c2738a4b0ec 2023-02-09 22:55:01,871 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123128d8d1ac6f1d45a3ac9ba2424f326b4f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123128d8d1ac6f1d45a3ac9ba2424f326b4f 2023-02-09 22:55:01,872 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019bd24789afb94b0dbfe98b5df5b54800 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019bd24789afb94b0dbfe98b5df5b54800 2023-02-09 22:55:01,873 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034168a377b4124c3091cefbe536a225ba to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034168a377b4124c3091cefbe536a225ba 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-09 22:55:01,873 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:01,905 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): e645e62e6f6942b9a9c124b648b04211#family1#compaction#73 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:02,722 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/459bf3f1711d45278bd5c7cfc05e1783 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/459bf3f1711d45278bd5c7cfc05e1783 2023-02-09 22:55:02,726 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e645e62e6f6942b9a9c124b648b04211/family1 of e645e62e6f6942b9a9c124b648b04211 into 459bf3f1711d45278bd5c7cfc05e1783(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:02,726 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:02,726 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675983301865; duration=0sec 2023-02-09 22:55:02,726 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:02,727 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058a7722f06b864554a4fec2d9a2043eb4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058a7722f06b864554a4fec2d9a2043eb4 2023-02-09 22:55:02,727 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:02,751 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2a048e12d2e448108deda00e8261c245 first=Optional[r00] last=Optional[r2] 2023-02-09 22:55:02,751 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2a048e12d2e448108deda00e8261c245}] 2023-02-09 22:55:02,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2a048e12d2e448108deda00e8261c245 for inclusion in e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:02,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-09 22:55:02,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:55:02,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:02,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2a048e12d2e448108deda00e8261c245 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__5mh4v5vmb5pgrrngd2agn3nnl776qs881f8voq86kljldboa7e3ksneli4obs7h9/family1/2a048e12d2e448108deda00e8261c245 2023-02-09 22:55:02,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__5mh4v5vmb5pgrrngd2agn3nnl776qs881f8voq86kljldboa7e3ksneli4obs7h9/family1/2a048e12d2e448108deda00e8261c245 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_ 2023-02-09 22:55:02,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__5mh4v5vmb5pgrrngd2agn3nnl776qs881f8voq86kljldboa7e3ksneli4obs7h9/family1/2a048e12d2e448108deda00e8261c245 into e645e62e6f6942b9a9c124b648b04211/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_ - updating store file list. 2023-02-09 22:55:02,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_ into e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:02,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__5mh4v5vmb5pgrrngd2agn3nnl776qs881f8voq86kljldboa7e3ksneli4obs7h9/family1/2a048e12d2e448108deda00e8261c245 into e645e62e6f6942b9a9c124b648b04211/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_) 2023-02-09 22:55:02,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__5mh4v5vmb5pgrrngd2agn3nnl776qs881f8voq86kljldboa7e3ksneli4obs7h9/family1/2a048e12d2e448108deda00e8261c245 2023-02-09 22:55:02,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-09 22:55:02,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-09 22:55:02,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): e645e62e6f6942b9a9c124b648b04211/family1 is initiating minor compaction (all files) 2023-02-09 22:55:02,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@60c95520; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:02,766 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e645e62e6f6942b9a9c124b648b04211/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:02,766 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/459bf3f1711d45278bd5c7cfc05e1783, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp, totalSize=388.1 K 2023-02-09 22:55:02,766 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 343c92148e5b412f85084fe02eb69c9b, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878839414 2023-02-09 22:55:02,767 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 459bf3f1711d45278bd5c7cfc05e1783, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621239414 2023-02-09 22:55:02,767 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 44b65702625942c8adf3b1d9f7afba07_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878839414 2023-02-09 22:55:02,767 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:02,768 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:02,769 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a6a96f09102a44258690468f2005de01 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a6a96f09102a44258690468f2005de01 2023-02-09 22:55:02,771 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512011bcf174282054ab8b5b64f960479b6d1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512011bcf174282054ab8b5b64f960479b6d1 2023-02-09 22:55:02,772 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058eabb98729144a4997ace49f8e1d35d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512058eabb98729144a4997ace49f8e1d35d2 2023-02-09 22:55:02,772 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-09 22:55:02,772 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-09 22:55:02,773 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:02,773 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:02,773 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:02,775 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:02,784 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.4050 sec 2023-02-09 22:55:02,786 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_] to archive 2023-02-09 22:55:02,787 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:02,789 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/3d56ed8ffe8a40f3a2e0dc2de147fe37 2023-02-09 22:55:02,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5d26ef0e7f5444e8ba211fdd2860bc4d 2023-02-09 22:55:02,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/501fc049e49943f99292135078139ab4 2023-02-09 22:55:02,793 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/52f81befbce0428d9462a039685b88a1 2023-02-09 22:55:02,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/78dd8009acf44db0b55ab9a24df20f7e 2023-02-09 22:55:02,795 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): e645e62e6f6942b9a9c124b648b04211#family1#compaction#74 average throughput is 12.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:02,796 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/1dc2b90bd6914228859c4d4bd9e1d30e 2023-02-09 22:55:02,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/4f55e8220e294337a019bbab095e9aab 2023-02-09 22:55:02,799 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/881ba73b214f42a08ab92901a21f0b07 2023-02-09 22:55:02,801 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/385b49f1e6064fae98a2d7311a2e2eaa 2023-02-09 22:55:02,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/c47fce6cdb754262a195362978d62df5_SeqId_44_ 2023-02-09 22:55:02,819 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/.tmp/family1/cf65ca8956c944eca7be3e4b7331fb63 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/cf65ca8956c944eca7be3e4b7331fb63 2023-02-09 22:55:02,825 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e645e62e6f6942b9a9c124b648b04211/family1 of e645e62e6f6942b9a9c124b648b04211 into cf65ca8956c944eca7be3e4b7331fb63(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:02,825 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:02,825 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1675983302765; duration=0sec 2023-02-09 22:55:02,825 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:02,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 22:55:02,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:02,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/459bf3f1711d45278bd5c7cfc05e1783, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_] to archive 2023-02-09 22:55:02,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:02,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/343c92148e5b412f85084fe02eb69c9b 2023-02-09 22:55:02,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/459bf3f1711d45278bd5c7cfc05e1783 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/459bf3f1711d45278bd5c7cfc05e1783 2023-02-09 22:55:02,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/44b65702625942c8adf3b1d9f7afba07_SeqId_47_ 2023-02-09 22:55:03,264 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 22:55:03,265 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:03,265 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 22:55:03,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 22:55:03,269 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE}] 2023-02-09 22:55:03,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE 2023-02-09 22:55:03,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:03,271 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983303270"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983303270"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983303270"}]},"ts":"1675983303270"} 2023-02-09 22:55:03,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:03,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing e645e62e6f6942b9a9c124b648b04211, disabling compactions & flushes 2023-02-09 22:55:03,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. after waiting 0 ms 2023-02-09 22:55:03,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,429 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 22:55:03,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for e645e62e6f6942b9a9c124b648b04211: Waiting for close lock at 1675983303424Running coprocessor pre-close hooks at 1675983303424Disabling compacts and flushes for region at 1675983303424Disabling writes for close at 1675983303424Writing region close event to WAL at 1675983303426 (+2 ms)Running coprocessor post-close hooks at 1675983303429 (+3 ms)Closed at 1675983303430 (+1 ms) 2023-02-09 22:55:03,430 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e645e62e6f6942b9a9c124b648b04211 to self. 2023-02-09 22:55:03,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,431 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=CLOSED 2023-02-09 22:55:03,432 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983303431"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983303431"}]},"ts":"1675983303431"} 2023-02-09 22:55:03,434 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-09 22:55:03,434 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026 in 161 msec 2023-02-09 22:55:03,434 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=true 2023-02-09 22:55:03,585 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:03,585 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983303585"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983303585"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983303585"}]},"ts":"1675983303585"} 2023-02-09 22:55:03,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:03,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e645e62e6f6942b9a9c124b648b04211, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:03,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:03,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,742 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,743 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:03,743 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:03,743 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e645e62e6f6942b9a9c124b648b04211 columnFamilyName family1 2023-02-09 22:55:03,748 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/cf65ca8956c944eca7be3e4b7331fb63 2023-02-09 22:55:03,748 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(310): Store=e645e62e6f6942b9a9c124b648b04211/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:03,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:03,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened e645e62e6f6942b9a9c124b648b04211; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59385465, jitterRate=-0.1150876134634018}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:55:03,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for e645e62e6f6942b9a9c124b648b04211: Running coprocessor pre-open hook at 1675983303741Writing region info on filesystem at 1675983303741Initializing all the Stores at 1675983303742 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675983303742Cleaning up temporary data from old regions at 1675983303750 (+8 ms)Cleaning up detritus from prior splits at 1675983303750Running coprocessor post-open hooks at 1675983303753 (+3 ms)Region opened successfully at 1675983303753 2023-02-09 22:55:03,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., pid=63, masterSystemTime=1675983303738 2023-02-09 22:55:03,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:03,755 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:03,755 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983303755"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983303755"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983303755"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983303755"}]},"ts":"1675983303755"} 2023-02-09 22:55:03,757 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-09 22:55:03,757 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026 in 170 msec 2023-02-09 22:55:03,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-09 22:55:03,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE in 488 msec 2023-02-09 22:55:03,760 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-09 22:55:03,760 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 492 msec 2023-02-09 22:55:03,761 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 905 msec 2023-02-09 22:55:03,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-09 22:55:03,763 INFO [Listener at localhost/33819] compactions.TestMobCompactor(278): alter status finished 2023-02-09 22:55:03,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:03,764 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:03,765 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:03,765 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:03,765 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983903765 2023-02-09 22:55:03,766 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:55:03,767 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 22:55:03,767 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:03,767 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x26bd9038 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:03,771 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ca5c5d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:03,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:03,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:03,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:03,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:03,771 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:04,594 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128774610cc2fca4b66944982927eec33aa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128774610cc2fca4b66944982927eec33aa 2023-02-09 22:55:04,595 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:04,607 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:04,608 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:04,615 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:04,616 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49394, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:04,624 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/beae1683c30b4bcfa6c70e289775eda8 first=Optional[r05] last=Optional[r8] 2023-02-09 22:55:04,625 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/beae1683c30b4bcfa6c70e289775eda8}] 2023-02-09 22:55:04,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/beae1683c30b4bcfa6c70e289775eda8 for inclusion in e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:04,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-09 22:55:04,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:55:04,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for e645e62e6f6942b9a9c124b648b04211: 2023-02-09 22:55:04,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/beae1683c30b4bcfa6c70e289775eda8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__6sidhjg8s68ujlebsml87r8sq3tvt88fqf4ac8ahvo9u5iisgsu0vu9v8jg95glq/family1/beae1683c30b4bcfa6c70e289775eda8 2023-02-09 22:55:04,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__6sidhjg8s68ujlebsml87r8sq3tvt88fqf4ac8ahvo9u5iisgsu0vu9v8jg95glq/family1/beae1683c30b4bcfa6c70e289775eda8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5f13a76f95fb43c7b1ef35e95da529d8_SeqId_53_ 2023-02-09 22:55:04,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__6sidhjg8s68ujlebsml87r8sq3tvt88fqf4ac8ahvo9u5iisgsu0vu9v8jg95glq/family1/beae1683c30b4bcfa6c70e289775eda8 into e645e62e6f6942b9a9c124b648b04211/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5f13a76f95fb43c7b1ef35e95da529d8_SeqId_53_ - updating store file list. 2023-02-09 22:55:04,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5f13a76f95fb43c7b1ef35e95da529d8_SeqId_53_ into e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:04,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__6sidhjg8s68ujlebsml87r8sq3tvt88fqf4ac8ahvo9u5iisgsu0vu9v8jg95glq/family1/beae1683c30b4bcfa6c70e289775eda8 into e645e62e6f6942b9a9c124b648b04211/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5f13a76f95fb43c7b1ef35e95da529d8_SeqId_53_) 2023-02-09 22:55:04,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__6sidhjg8s68ujlebsml87r8sq3tvt88fqf4ac8ahvo9u5iisgsu0vu9v8jg95glq/family1/beae1683c30b4bcfa6c70e289775eda8 2023-02-09 22:55:04,639 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:04,640 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:04,641 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112004144c5208e94700b3839d87b7d71510 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112004144c5208e94700b3839d87b7d71510 2023-02-09 22:55:04,642 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112826a4b8457fb84b6d8b3157dfe5523132 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112826a4b8457fb84b6d8b3157dfe5523132 2023-02-09 22:55:04,643 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-09 22:55:04,643 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 22:55:04,643 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:04,643 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:04,643 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:04,643 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:04,643 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:04,644 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:04,645 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 880 msec 2023-02-09 22:55:04,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$12(2441): Client=jenkins//172.31.2.10 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-09 22:55:04,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:04,705 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 22:55:04,705 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-09 22:55:04,705 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-09 22:55:04,707 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-09 22:55:04,709 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE}] 2023-02-09 22:55:04,709 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE 2023-02-09 22:55:04,710 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:04,710 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983304710"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983304710"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983304710"}]},"ts":"1675983304710"} 2023-02-09 22:55:04,711 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:04,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:04,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing e645e62e6f6942b9a9c124b648b04211, disabling compactions & flushes 2023-02-09 22:55:04,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:04,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:04,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. after waiting 0 ms 2023-02-09 22:55:04,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:04,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 22:55:04,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:04,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for e645e62e6f6942b9a9c124b648b04211: Waiting for close lock at 1675983304863Running coprocessor pre-close hooks at 1675983304863Disabling compacts and flushes for region at 1675983304863Disabling writes for close at 1675983304863Writing region close event to WAL at 1675983304867 (+4 ms)Running coprocessor post-close hooks at 1675983304871 (+4 ms)Closed at 1675983304871 2023-02-09 22:55:04,871 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e645e62e6f6942b9a9c124b648b04211 to self. 2023-02-09 22:55:04,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:04,872 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=CLOSED 2023-02-09 22:55:04,873 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983304872"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983304872"}]},"ts":"1675983304872"} 2023-02-09 22:55:04,875 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-09 22:55:04,875 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026 in 163 msec 2023-02-09 22:55:04,876 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=true 2023-02-09 22:55:05,026 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:05,026 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983305026"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983305026"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983305026"}]},"ts":"1675983305026"} 2023-02-09 22:55:05,028 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:05,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:05,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e645e62e6f6942b9a9c124b648b04211, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:05,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:05,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,183 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,184 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:05,184 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1 2023-02-09 22:55:05,184 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e645e62e6f6942b9a9c124b648b04211 columnFamilyName family1 2023-02-09 22:55:05,188 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/5f13a76f95fb43c7b1ef35e95da529d8_SeqId_53_ 2023-02-09 22:55:05,191 DEBUG [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/family1/cf65ca8956c944eca7be3e4b7331fb63 2023-02-09 22:55:05,191 INFO [StoreOpener-e645e62e6f6942b9a9c124b648b04211-1] regionserver.HStore(310): Store=e645e62e6f6942b9a9c124b648b04211/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:05,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:55:05,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened e645e62e6f6942b9a9c124b648b04211; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66493064, jitterRate=-0.009176135063171387}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:55:05,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for e645e62e6f6942b9a9c124b648b04211: Running coprocessor pre-open hook at 1675983305182Writing region info on filesystem at 1675983305182Initializing all the Stores at 1675983305183 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1675983305183Cleaning up temporary data from old regions at 1675983305193 (+10 ms)Cleaning up detritus from prior splits at 1675983305194 (+1 ms)Running coprocessor post-open hooks at 1675983305196 (+2 ms)Region opened successfully at 1675983305196 2023-02-09 22:55:05,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211., pid=69, masterSystemTime=1675983305179 2023-02-09 22:55:05,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:05,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:55:05,198 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e645e62e6f6942b9a9c124b648b04211, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:05,198 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675983305198"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983305198"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983305198"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983305198"}]},"ts":"1675983305198"} 2023-02-09 22:55:05,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-09 22:55:05,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure e645e62e6f6942b9a9c124b648b04211, server=jenkins-hbase9.apache.org,39093,1675983215026 in 171 msec 2023-02-09 22:55:05,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-09 22:55:05,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e645e62e6f6942b9a9c124b648b04211, REOPEN/MOVE in 491 msec 2023-02-09 22:55:05,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 22:55:05,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 495 msec 2023-02-09 22:55:05,204 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 507 msec 2023-02-09 22:55:05,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-09 22:55:05,206 INFO [Listener at localhost/33819] compactions.TestMobCompactor(278): alter status finished 2023-02-09 22:55:05,207 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-09 22:55:05,208 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:05,208 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:05,209 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:05,209 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983905209 2023-02-09 22:55:05,209 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:55:05,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-09 22:55:05,210 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:05,211 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x0bb8c256 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:05,219 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e278f00, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-09 22:55:05,219 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-09 22:55:05,219 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:05,220 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-09 22:55:05,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 13 msec 2023-02-09 22:55:05,272 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=240 (was 246), OpenFileDescriptor=609 (was 622), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 146), ProcessCount=168 (was 168), AvailableMemoryMB=10750 (was 10837) 2023-02-09 22:55:05,284 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=240, OpenFileDescriptor=609, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=168, AvailableMemoryMB=10750 2023-02-09 22:55:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 22:55:05,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:05,288 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:05,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-09 22:55:05,289 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:05,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:05,290 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:05,290 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a empty. 2023-02-09 22:55:05,291 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:05,291 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 22:55:05,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:05,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:05,701 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:05,703 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => b5522bfe7bbacd2f99095ab3499bb79a, NAME => 'testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:05,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing b5522bfe7bbacd2f99095ab3499bb79a, disabling compactions & flushes 2023-02-09 22:55:06,108 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. after waiting 0 ms 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,108 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,108 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for b5522bfe7bbacd2f99095ab3499bb79a: Waiting for close lock at 1675983306108Disabling compacts and flushes for region at 1675983306108Disabling writes for close at 1675983306108Writing region close event to WAL at 1675983306108Closed at 1675983306108 2023-02-09 22:55:06,110 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:06,111 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983306111"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983306111"}]},"ts":"1675983306111"} 2023-02-09 22:55:06,112 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:06,113 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:06,113 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983306113"}]},"ts":"1675983306113"} 2023-02-09 22:55:06,114 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-09 22:55:06,116 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, ASSIGN}] 2023-02-09 22:55:06,118 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, ASSIGN 2023-02-09 22:55:06,118 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:06,269 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=b5522bfe7bbacd2f99095ab3499bb79a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:06,269 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983306269"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983306269"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983306269"}]},"ts":"1675983306269"} 2023-02-09 22:55:06,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure b5522bfe7bbacd2f99095ab3499bb79a, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:06,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:06,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b5522bfe7bbacd2f99095ab3499bb79a, NAME => 'testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:06,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-09 22:55:06,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-09 22:55:06,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:06,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,428 INFO [StoreOpener-b5522bfe7bbacd2f99095ab3499bb79a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,429 DEBUG [StoreOpener-b5522bfe7bbacd2f99095ab3499bb79a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1 2023-02-09 22:55:06,429 DEBUG [StoreOpener-b5522bfe7bbacd2f99095ab3499bb79a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1 2023-02-09 22:55:06,430 INFO [StoreOpener-b5522bfe7bbacd2f99095ab3499bb79a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5522bfe7bbacd2f99095ab3499bb79a columnFamilyName f1 2023-02-09 22:55:06,431 INFO [StoreOpener-b5522bfe7bbacd2f99095ab3499bb79a-1] regionserver.HStore(310): Store=b5522bfe7bbacd2f99095ab3499bb79a/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:06,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:06,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:06,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened b5522bfe7bbacd2f99095ab3499bb79a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62500435, jitterRate=-0.06867094337940216}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:55:06,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for b5522bfe7bbacd2f99095ab3499bb79a: Running coprocessor pre-open hook at 1675983306427Writing region info on filesystem at 1675983306427Initializing all the Stores at 1675983306428 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983306428Cleaning up temporary data from old regions at 1675983306433 (+5 ms)Cleaning up detritus from prior splits at 1675983306433Running coprocessor post-open hooks at 1675983306436 (+3 ms)Region opened successfully at 1675983306436 2023-02-09 22:55:06,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a., pid=73, masterSystemTime=1675983306422 2023-02-09 22:55:06,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:06,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=b5522bfe7bbacd2f99095ab3499bb79a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:06,439 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983306438"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983306438"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983306438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983306438"}]},"ts":"1675983306438"} 2023-02-09 22:55:06,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 22:55:06,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure b5522bfe7bbacd2f99095ab3499bb79a, server=jenkins-hbase9.apache.org,39093,1675983215026 in 169 msec 2023-02-09 22:55:06,442 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-09 22:55:06,442 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, ASSIGN in 325 msec 2023-02-09 22:55:06,442 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:06,443 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983306442"}]},"ts":"1675983306442"} 2023-02-09 22:55:06,444 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-09 22:55:06,445 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:06,446 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1590 sec 2023-02-09 22:55:06,946 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-09 22:55:07,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-09 22:55:07,394 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-09 22:55:07,394 DEBUG [Listener at localhost/33819] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-09 22:55:07,396 INFO [Listener at localhost/33819] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 22:55:07,399 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-09 22:55:07,399 INFO [Listener at localhost/33819] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 22:55:07,399 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-09 22:55:07,403 INFO [Listener at localhost/33819] regionserver.HRegion(2744): Flushing b5522bfe7bbacd2f99095ab3499bb79a 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 22:55:07,814 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302098c42b587005b42cca31217e168b01c0c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302098c42b587005b42cca31217e168b01c0c 2023-02-09 22:55:08,220 INFO [Listener at localhost/33819] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp/f1/265e48b0db3f4c108e0a42407fa4deae 2023-02-09 22:55:08,225 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp/f1/265e48b0db3f4c108e0a42407fa4deae as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae 2023-02-09 22:55:08,228 INFO [Listener at localhost/33819] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae, entries=1, sequenceid=5, filesize=4.9 K 2023-02-09 22:55:08,229 INFO [Listener at localhost/33819] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for b5522bfe7bbacd2f99095ab3499bb79a in 826ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:08,230 DEBUG [Listener at localhost/33819] regionserver.HRegion(2445): Flush status journal for b5522bfe7bbacd2f99095ab3499bb79a: 2023-02-09 22:55:08,231 INFO [Listener at localhost/33819] regionserver.HRegion(2744): Flushing b5522bfe7bbacd2f99095ab3499bb79a 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-09 22:55:08,642 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209e06981beb36f414f8f06f4807776a52c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209e06981beb36f414f8f06f4807776a52c 2023-02-09 22:55:08,960 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:09,047 INFO [Listener at localhost/33819] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp/f1/efc298bfaad9468fb0f443108a0b09a8 2023-02-09 22:55:09,052 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp/f1/efc298bfaad9468fb0f443108a0b09a8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8 2023-02-09 22:55:09,055 INFO [Listener at localhost/33819] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8, entries=1, sequenceid=9, filesize=4.9 K 2023-02-09 22:55:09,056 INFO [Listener at localhost/33819] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for b5522bfe7bbacd2f99095ab3499bb79a in 825ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:09,056 DEBUG [Listener at localhost/33819] regionserver.HRegion(2445): Flush status journal for b5522bfe7bbacd2f99095ab3499bb79a: 2023-02-09 22:55:09,057 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-09 22:55:09,059 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 22:55:09,059 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 22:55:09,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:09,060 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983909060 2023-02-09 22:55:09,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:55:09,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-09 22:55:09,060 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-09 22:55:09,061 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x0fc4a5f1 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:09,065 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d796486, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:09,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 22:55:09,109 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,109 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,160 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,160 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,210 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,211 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,261 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,261 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,312 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,312 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,363 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,363 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,414 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,414 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,465 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,465 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,516 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,516 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,567 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,567 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,618 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,618 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,669 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,669 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,720 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,720 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,771 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,771 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,821 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,822 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,872 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,872 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,889 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302096e46c627eb5d41b3b2488e2467fd2802 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302096e46c627eb5d41b3b2488e2467fd2802 2023-02-09 22:55:09,889 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 22:55:09,902 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:09,903 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33094, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:09,908 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:09,910 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:49410, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:09,917 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/29ace3a09dbd4c7c9eef7619645f9db1 first=Optional[r1] last=Optional[r2] 2023-02-09 22:55:09,918 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{f1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/29ace3a09dbd4c7c9eef7619645f9db1}] 2023-02-09 22:55:09,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/29ace3a09dbd4c7c9eef7619645f9db1 for inclusion in b5522bfe7bbacd2f99095ab3499bb79a/f1 2023-02-09 22:55:09,923 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,923 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-09 22:55:09,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:55:09,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for b5522bfe7bbacd2f99095ab3499bb79a: 2023-02-09 22:55:09,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/f1/29ace3a09dbd4c7c9eef7619645f9db1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testGetAfterCompaction__32fjqod8200subv7k0i0kop3bkunkfs4dkqc545at3rss4b7ro80ir6m6992nb5d/f1/29ace3a09dbd4c7c9eef7619645f9db1 2023-02-09 22:55:09,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testGetAfterCompaction__32fjqod8200subv7k0i0kop3bkunkfs4dkqc545at3rss4b7ro80ir6m6992nb5d/f1/29ace3a09dbd4c7c9eef7619645f9db1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_ 2023-02-09 22:55:09,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testGetAfterCompaction__32fjqod8200subv7k0i0kop3bkunkfs4dkqc545at3rss4b7ro80ir6m6992nb5d/f1/29ace3a09dbd4c7c9eef7619645f9db1 into b5522bfe7bbacd2f99095ab3499bb79a/f1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_ - updating store file list. 2023-02-09 22:55:09,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_ into b5522bfe7bbacd2f99095ab3499bb79a/f1 2023-02-09 22:55:09,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testGetAfterCompaction__32fjqod8200subv7k0i0kop3bkunkfs4dkqc545at3rss4b7ro80ir6m6992nb5d/f1/29ace3a09dbd4c7c9eef7619645f9db1 into b5522bfe7bbacd2f99095ab3499bb79a/f1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_) 2023-02-09 22:55:09,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testGetAfterCompaction__32fjqod8200subv7k0i0kop3bkunkfs4dkqc545at3rss4b7ro80ir6m6992nb5d/f1/29ace3a09dbd4c7c9eef7619645f9db1 2023-02-09 22:55:09,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(1912): b5522bfe7bbacd2f99095ab3499bb79a/f1 is initiating minor compaction 2023-02-09 22:55:09,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@27c41e36; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:09,930 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b5522bfe7bbacd2f99095ab3499bb79a/f1 in testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:09,930 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp, totalSize=10.0 K 2023-02-09 22:55:09,930 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting efc298bfaad9468fb0f443108a0b09a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:55:09,930 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:55:09,932 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testGetAfterCompaction' column='f1' 2023-02-09 22:55:09,932 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:09,935 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302098c42b587005b42cca31217e168b01c0c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302098c42b587005b42cca31217e168b01c0c 2023-02-09 22:55:09,936 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209e06981beb36f414f8f06f4807776a52c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230209e06981beb36f414f8f06f4807776a52c 2023-02-09 22:55:09,936 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-09 22:55:09,936 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 22:55:09,937 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testGetAfterCompaction' and column='f1' 2023-02-09 22:55:09,937 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-09 22:55:09,937 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:55:09,937 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): b5522bfe7bbacd2f99095ab3499bb79a#f1#compaction#77 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:09,938 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-09 22:55:09,939 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 880 msec 2023-02-09 22:55:09,974 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,974 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:09,974 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:09,978 DEBUG [Listener at localhost/33819] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 22:55:09,982 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33102, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 22:55:09,982 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39093] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:09,982 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-09 22:55:09,982 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 22:55:09,982 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39093] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 22:55:09,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39093] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. because compaction request was cancelled 2023-02-09 22:55:10,034 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,034 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,034 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,085 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,085 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,085 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,136 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,136 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,136 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,187 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,187 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,187 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,238 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,238 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,238 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,289 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,289 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,289 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,340 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,340 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,340 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,390 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,391 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,391 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,441 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,442 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,442 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,492 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,492 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,492 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,543 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,543 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,543 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,594 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,594 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,594 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,645 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,645 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,645 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,696 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,696 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,696 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,747 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,747 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675983309885; access_time=1675983309077; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-09 22:55:10,747 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983309047; access_time=1675983308233; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,751 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/.tmp/f1/ca78c9f80be64cb2b274c7198d84b5d6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/ca78c9f80be64cb2b274c7198d84b5d6 2023-02-09 22:55:10,756 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in b5522bfe7bbacd2f99095ab3499bb79a/f1 of b5522bfe7bbacd2f99095ab3499bb79a into ca78c9f80be64cb2b274c7198d84b5d6(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:10,756 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b5522bfe7bbacd2f99095ab3499bb79a: 2023-02-09 22:55:10,756 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1675983309930; duration=0sec 2023-02-09 22:55:10,756 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:10,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_] to archive 2023-02-09 22:55:10,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:10,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/efc298bfaad9468fb0f443108a0b09a8 2023-02-09 22:55:10,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/cc0e38305c2346ef83e0f974e5e7fba1_SeqId_12_ 2023-02-09 22:55:10,797 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675983308219; access_time=1675983307405; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,797 INFO [Listener at localhost/33819] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/ca78c9f80be64cb2b274c7198d84b5d6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675983310746; access_time=1675983309935; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-09 22:55:10,914 INFO [Listener at localhost/33819] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-09 22:55:10,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testGetAfterCompaction 2023-02-09 22:55:10,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:10,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983310929"}]},"ts":"1675983310929"} 2023-02-09 22:55:10,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 22:55:10,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-09 22:55:10,932 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-09 22:55:10,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, UNASSIGN}] 2023-02-09 22:55:10,934 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, UNASSIGN 2023-02-09 22:55:10,935 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=b5522bfe7bbacd2f99095ab3499bb79a, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:10,935 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983310935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983310935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983310935"}]},"ts":"1675983310935"} 2023-02-09 22:55:10,936 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure b5522bfe7bbacd2f99095ab3499bb79a, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:11,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 22:55:11,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:11,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing b5522bfe7bbacd2f99095ab3499bb79a, disabling compactions & flushes 2023-02-09 22:55:11,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:11,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:11,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. after waiting 0 ms 2023-02-09 22:55:11,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:11,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 22:55:11,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-09 22:55:11,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a. 2023-02-09 22:55:11,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for b5522bfe7bbacd2f99095ab3499bb79a: Waiting for close lock at 1675983311088Running coprocessor pre-close hooks at 1675983311088Disabling compacts and flushes for region at 1675983311088Disabling writes for close at 1675983311088Writing region close event to WAL at 1675983311089 (+1 ms)Running coprocessor post-close hooks at 1675983311092 (+3 ms)Closed at 1675983311093 (+1 ms) 2023-02-09 22:55:11,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:11,095 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=b5522bfe7bbacd2f99095ab3499bb79a, regionState=CLOSED 2023-02-09 22:55:11,095 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675983311095"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983311095"}]},"ts":"1675983311095"} 2023-02-09 22:55:11,097 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-09 22:55:11,097 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure b5522bfe7bbacd2f99095ab3499bb79a, server=jenkins-hbase9.apache.org,39093,1675983215026 in 160 msec 2023-02-09 22:55:11,098 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-09 22:55:11,098 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=b5522bfe7bbacd2f99095ab3499bb79a, UNASSIGN in 164 msec 2023-02-09 22:55:11,099 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983311099"}]},"ts":"1675983311099"} 2023-02-09 22:55:11,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-09 22:55:11,106 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-09 22:55:11,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 186 msec 2023-02-09 22:55:11,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-09 22:55:11,232 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-09 22:55:11,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$5(2215): Client=jenkins//172.31.2.10 delete testGetAfterCompaction 2023-02-09 22:55:11,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,244 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,245 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 22:55:11,248 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:11,250 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1, FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/recovered.edits] 2023-02-09 22:55:11,254 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/265e48b0db3f4c108e0a42407fa4deae 2023-02-09 22:55:11,255 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/ca78c9f80be64cb2b274c7198d84b5d6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/f1/ca78c9f80be64cb2b274c7198d84b5d6 2023-02-09 22:55:11,257 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/recovered.edits/15.seqid to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a/recovered.edits/15.seqid 2023-02-09 22:55:11,258 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testGetAfterCompaction/b5522bfe7bbacd2f99095ab3499bb79a 2023-02-09 22:55:11,258 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-09 22:55:11,258 DEBUG [PEWorker-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 22:55:11,259 DEBUG [PEWorker-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-09 22:55:11,261 DEBUG [PEWorker-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302096e46c627eb5d41b3b2488e2467fd2802 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302096e46c627eb5d41b3b2488e2467fd2802 2023-02-09 22:55:11,261 DEBUG [PEWorker-2] backup.HFileArchiver(596): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-09 22:55:11,263 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,284 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-09 22:55:11,286 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-09 22:55:11,287 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,287 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-09 22:55:11,287 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983311287"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:11,289 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 22:55:11,289 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b5522bfe7bbacd2f99095ab3499bb79a, NAME => 'testGetAfterCompaction,,1675983305286.b5522bfe7bbacd2f99095ab3499bb79a.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 22:55:11,289 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-09 22:55:11,289 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675983311289"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:11,290 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-09 22:55:11,292 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-09 22:55:11,293 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 54 msec 2023-02-09 22:55:11,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-09 22:55:11,347 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-09 22:55:11,360 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=244 (was 240) - Thread LEAK? -, OpenFileDescriptor=615 (was 609) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 112), ProcessCount=168 (was 168), AvailableMemoryMB=10747 (was 10750) 2023-02-09 22:55:11,371 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=244, OpenFileDescriptor=615, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=168, AvailableMemoryMB=10746 2023-02-09 22:55:11,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 22:55:11,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-09 22:55:11,375 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:11,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-09 22:55:11,376 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:11,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 empty. 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 empty. 2023-02-09 22:55:11,378 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 empty. 2023-02-09 22:55:11,379 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:11,379 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:11,379 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:11,379 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-09 22:55:11,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:11,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:11,786 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:11,787 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 644cc6a94a1dc8fa8754303752b40531, NAME => 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:11,788 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => d030c0c1080b19b33ee54050ad4ea0a2, NAME => 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:11,788 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1bc7ec556ab621014c7a07a6e844bff2, NAME => 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:11,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:12,193 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,194 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 644cc6a94a1dc8fa8754303752b40531, disabling compactions & flushes 2023-02-09 22:55:12,194 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,194 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,194 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. after waiting 0 ms 2023-02-09 22:55:12,194 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,194 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,194 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 644cc6a94a1dc8fa8754303752b40531: Waiting for close lock at 1675983312194Disabling compacts and flushes for region at 1675983312194Disabling writes for close at 1675983312194Writing region close event to WAL at 1675983312194Closed at 1675983312194 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing d030c0c1080b19b33ee54050ad4ea0a2, disabling compactions & flushes 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 1bc7ec556ab621014c7a07a6e844bff2, disabling compactions & flushes 2023-02-09 22:55:12,195 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,195 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. after waiting 0 ms 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. after waiting 0 ms 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,195 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,195 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,195 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,196 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for d030c0c1080b19b33ee54050ad4ea0a2: Waiting for close lock at 1675983312195Disabling compacts and flushes for region at 1675983312195Disabling writes for close at 1675983312195Writing region close event to WAL at 1675983312195Closed at 1675983312195 2023-02-09 22:55:12,196 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 1bc7ec556ab621014c7a07a6e844bff2: Waiting for close lock at 1675983312195Disabling compacts and flushes for region at 1675983312195Disabling writes for close at 1675983312195Writing region close event to WAL at 1675983312195Closed at 1675983312195 2023-02-09 22:55:12,197 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:12,198 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983312198"}]},"ts":"1675983312198"} 2023-02-09 22:55:12,198 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983312198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983312198"}]},"ts":"1675983312198"} 2023-02-09 22:55:12,198 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983312198"}]},"ts":"1675983312198"} 2023-02-09 22:55:12,200 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 22:55:12,201 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:12,201 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983312201"}]},"ts":"1675983312201"} 2023-02-09 22:55:12,202 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 22:55:12,206 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN}] 2023-02-09 22:55:12,208 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN 2023-02-09 22:55:12,208 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN 2023-02-09 22:55:12,208 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN 2023-02-09 22:55:12,208 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:12,208 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:12,208 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:12,360 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,360 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,360 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,360 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312360"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983312360"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983312360"}]},"ts":"1675983312360"} 2023-02-09 22:55:12,360 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983312360"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983312360"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983312360"}]},"ts":"1675983312360"} 2023-02-09 22:55:12,361 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312360"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983312360"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983312360"}]},"ts":"1675983312360"} 2023-02-09 22:55:12,362 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:12,362 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:12,364 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:12,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:12,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1bc7ec556ab621014c7a07a6e844bff2, NAME => 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 22:55:12,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,519 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,520 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:12,520 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:12,520 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bc7ec556ab621014c7a07a6e844bff2 columnFamilyName family1 2023-02-09 22:55:12,522 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(310): Store=1bc7ec556ab621014c7a07a6e844bff2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,522 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,523 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2 2023-02-09 22:55:12,523 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2 2023-02-09 22:55:12,524 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bc7ec556ab621014c7a07a6e844bff2 columnFamilyName family2 2023-02-09 22:55:12,525 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(310): Store=1bc7ec556ab621014c7a07a6e844bff2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:12,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:12,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:12,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 1bc7ec556ab621014c7a07a6e844bff2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62907702, jitterRate=-0.06260219216346741}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:12,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 1bc7ec556ab621014c7a07a6e844bff2: Running coprocessor pre-open hook at 1675983312518Writing region info on filesystem at 1675983312518Initializing all the Stores at 1675983312518Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312518Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312518Cleaning up temporary data from old regions at 1675983312526 (+8 ms)Cleaning up detritus from prior splits at 1675983312527 (+1 ms)Running coprocessor post-open hooks at 1675983312531 (+4 ms)Region opened successfully at 1675983312531 2023-02-09 22:55:12,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., pid=83, masterSystemTime=1675983312514 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:12,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 644cc6a94a1dc8fa8754303752b40531, NAME => 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 22:55:12,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312533"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983312533"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983312533"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983312533"}]},"ts":"1675983312533"} 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,534 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,535 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:12,536 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:12,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 22:55:12,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 172 msec 2023-02-09 22:55:12,536 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 644cc6a94a1dc8fa8754303752b40531 columnFamilyName family1 2023-02-09 22:55:12,537 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN in 331 msec 2023-02-09 22:55:12,537 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(310): Store=644cc6a94a1dc8fa8754303752b40531/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,537 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,538 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2 2023-02-09 22:55:12,538 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2 2023-02-09 22:55:12,538 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 644cc6a94a1dc8fa8754303752b40531 columnFamilyName family2 2023-02-09 22:55:12,539 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(310): Store=644cc6a94a1dc8fa8754303752b40531/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:12,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:12,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:12,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 644cc6a94a1dc8fa8754303752b40531; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68177209, jitterRate=0.015919581055641174}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:12,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 644cc6a94a1dc8fa8754303752b40531: Running coprocessor pre-open hook at 1675983312533Writing region info on filesystem at 1675983312533Initializing all the Stores at 1675983312534 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312534Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312534Cleaning up temporary data from old regions at 1675983312541 (+7 ms)Cleaning up detritus from prior splits at 1675983312541Running coprocessor post-open hooks at 1675983312544 (+3 ms)Region opened successfully at 1675983312544 2023-02-09 22:55:12,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., pid=85, masterSystemTime=1675983312514 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:12,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d030c0c1080b19b33ee54050ad4ea0a2, NAME => 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 22:55:12,546 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:12,546 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983312546"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983312546"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983312546"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983312546"}]},"ts":"1675983312546"} 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,547 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,548 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:12,548 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:12,549 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d030c0c1080b19b33ee54050ad4ea0a2 columnFamilyName family1 2023-02-09 22:55:12,549 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-02-09 22:55:12,549 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026 in 184 msec 2023-02-09 22:55:12,550 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(310): Store=d030c0c1080b19b33ee54050ad4ea0a2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,550 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN in 343 msec 2023-02-09 22:55:12,550 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,551 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2 2023-02-09 22:55:12,551 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2 2023-02-09 22:55:12,551 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d030c0c1080b19b33ee54050ad4ea0a2 columnFamilyName family2 2023-02-09 22:55:12,552 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(310): Store=d030c0c1080b19b33ee54050ad4ea0a2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:12,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:12,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:12,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:12,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened d030c0c1080b19b33ee54050ad4ea0a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63774131, jitterRate=-0.049691393971443176}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:12,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for d030c0c1080b19b33ee54050ad4ea0a2: Running coprocessor pre-open hook at 1675983312546Writing region info on filesystem at 1675983312546Initializing all the Stores at 1675983312547 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312547Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983312547Cleaning up temporary data from old regions at 1675983312553 (+6 ms)Cleaning up detritus from prior splits at 1675983312554 (+1 ms)Running coprocessor post-open hooks at 1675983312557 (+3 ms)Region opened successfully at 1675983312557 2023-02-09 22:55:12,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., pid=84, masterSystemTime=1675983312514 2023-02-09 22:55:12,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:12,559 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:12,559 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983312559"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983312559"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983312559"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983312559"}]},"ts":"1675983312559"} 2023-02-09 22:55:12,561 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-09 22:55:12,562 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 198 msec 2023-02-09 22:55:12,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-09 22:55:12,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN in 357 msec 2023-02-09 22:55:12,563 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:12,563 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983312563"}]},"ts":"1675983312563"} 2023-02-09 22:55:12,564 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 22:55:12,566 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:12,567 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.1930 sec 2023-02-09 22:55:12,947 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-09 22:55:13,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 22:55:13,481 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-09 22:55:13,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:13,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:13,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:13,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:13,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 22:55:13,494 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 22:55:13,494 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:13,494 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 22:55:13,494 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 22:55:13,494 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:13,495 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:13,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:13,496 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:13,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:13,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:13,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:13,496 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:13,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:13,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:13,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:13,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:13,497 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 22:55:13,497 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 22:55:13,497 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 22:55:13,497 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:13,497 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 22:55:13,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:13,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:13,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:13,497 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:13,497 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:13,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:13,498 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 1bc7ec556ab621014c7a07a6e844bff2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:13,497 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:13,498 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:13,499 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 644cc6a94a1dc8fa8754303752b40531 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:13,499 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d030c0c1080b19b33ee54050ad4ea0a2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:13,913 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 2023-02-09 22:55:13,913 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc 2023-02-09 22:55:13,913 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 2023-02-09 22:55:14,319 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/c85ef1364b9e4b06abc3c6c00f1d96ee 2023-02-09 22:55:14,319 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/4e7a9faf25f04cfea21bb52c9ed5615a 2023-02-09 22:55:14,320 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/55831fc6ce994a5cb81893af3b5f34fd 2023-02-09 22:55:14,741 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0 2023-02-09 22:55:14,741 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a 2023-02-09 22:55:14,745 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d 2023-02-09 22:55:14,749 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/e90eb10934884f2592db3b80bd585a91 2023-02-09 22:55:14,753 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/55831fc6ce994a5cb81893af3b5f34fd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd 2023-02-09 22:55:14,756 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:14,757 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/e90eb10934884f2592db3b80bd585a91 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91 2023-02-09 22:55:14,760 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:14,761 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d030c0c1080b19b33ee54050ad4ea0a2 in 1262ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:14,761 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:14,761 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:14,762 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:15,146 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/109596cc15d14cfea976ed9bedf2377f 2023-02-09 22:55:15,146 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba 2023-02-09 22:55:15,150 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/c85ef1364b9e4b06abc3c6c00f1d96ee as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee 2023-02-09 22:55:15,150 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/4e7a9faf25f04cfea21bb52c9ed5615a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a 2023-02-09 22:55:15,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:15,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:15,154 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba 2023-02-09 22:55:15,154 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/109596cc15d14cfea976ed9bedf2377f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f 2023-02-09 22:55:15,158 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:15,158 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:15,159 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1bc7ec556ab621014c7a07a6e844bff2 in 1661ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:15,159 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 644cc6a94a1dc8fa8754303752b40531 in 1660ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:15,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:15,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:15,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:15,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 22:55:15,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 22:55:15,160 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:15,160 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:15,161 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,161 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:15,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 22:55:15,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50e7903f[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:15,162 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 22:55:15,162 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,164 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,164 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 22:55:15,164 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 22:55:15,164 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,164 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:15,164 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:15,165 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 22:55:15,165 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,165 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:15,165 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 22:55:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:15,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 22:55:15,167 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 22:55:15,167 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:15,167 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:15,167 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 22:55:15,167 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:15,168 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,168 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:15,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:15,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,168 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:15,169 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:15,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,170 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:15,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:15,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,171 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:15,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 22:55:15,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:15,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:15,172 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:15,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:15,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:15,172 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:15,272 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:15,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:15,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:15,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:15,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:15,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:15,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:15,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 22:55:15,278 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 22:55:15,278 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:15,278 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 22:55:15,278 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 22:55:15,279 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,279 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:15,281 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,281 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:15,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:15,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:15,281 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:15,281 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:15,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:15,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:15,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 22:55:15,282 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 22:55:15,282 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 22:55:15,282 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 22:55:15,282 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:15,282 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:15,283 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:15,283 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 644cc6a94a1dc8fa8754303752b40531 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:15,283 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:15,283 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:15,283 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 1bc7ec556ab621014c7a07a6e844bff2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:15,283 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d030c0c1080b19b33ee54050ad4ea0a2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:15,705 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 2023-02-09 22:55:15,705 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b 2023-02-09 22:55:15,705 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 2023-02-09 22:55:16,111 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/978c8825afba40b3a59aa1565689d9c0 2023-02-09 22:55:16,111 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/6caab206f3eb4f499669f0c3f437a6b0 2023-02-09 22:55:16,111 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/cd51f7493e8d4aa19d1ec014571098c0 2023-02-09 22:55:16,528 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6 2023-02-09 22:55:16,532 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a 2023-02-09 22:55:16,532 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491 2023-02-09 22:55:16,594 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:16,933 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/94ad938e948e4b0eb64085e2a5d251d4 2023-02-09 22:55:16,937 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/5dd6f138a437444facbaa6253289b6f5 2023-02-09 22:55:16,937 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/74769bcc3f08490d9e45153b83bc91a7 2023-02-09 22:55:16,937 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/6caab206f3eb4f499669f0c3f437a6b0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0 2023-02-09 22:55:16,941 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:16,941 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/cd51f7493e8d4aa19d1ec014571098c0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0 2023-02-09 22:55:16,941 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/978c8825afba40b3a59aa1565689d9c0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0 2023-02-09 22:55:16,942 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/94ad938e948e4b0eb64085e2a5d251d4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4 2023-02-09 22:55:16,945 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:16,945 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:16,946 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/74769bcc3f08490d9e45153b83bc91a7 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7 2023-02-09 22:55:16,946 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:16,946 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/5dd6f138a437444facbaa6253289b6f5 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5 2023-02-09 22:55:16,946 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 644cc6a94a1dc8fa8754303752b40531 in 1663ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:16,947 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:16,947 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:16,947 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:16,949 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:16,950 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:16,950 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1bc7ec556ab621014c7a07a6e844bff2 in 1667ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:16,950 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d030c0c1080b19b33ee54050ad4ea0a2 in 1667ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:16,950 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:16,950 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:16,950 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:16,950 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 22:55:16,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 22:55:16,952 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:16,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:16,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 22:55:16,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9db9070[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:16,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 22:55:16,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,955 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,955 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 22:55:16,955 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 22:55:16,955 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:16,955 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,955 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:16,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 22:55:16,956 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:16,956 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 22:55:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 22:55:16,958 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 22:55:16,959 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:16,959 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:16,959 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 22:55:16,959 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:16,963 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,963 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:16,963 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:16,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:16,964 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:16,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,965 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:16,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,966 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,966 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:16,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 22:55:16,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:16,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:16,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:16,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 22:55:16,967 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:16,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:16,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:16,967 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:17,067 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:17,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:17,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:17,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:17,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:17,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:17,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:17,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 22:55:17,073 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 22:55:17,073 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:17,073 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 22:55:17,073 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 22:55:17,074 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:17,074 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:17,075 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:17,075 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:17,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:17,075 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:17,075 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:17,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:17,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:17,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:17,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 22:55:17,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 22:55:17,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 22:55:17,076 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 22:55:17,076 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:17,076 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 22:55:17,076 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:17,076 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:17,076 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:17,076 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:17,076 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:17,077 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 644cc6a94a1dc8fa8754303752b40531 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:17,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:17,076 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:17,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:17,077 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 1bc7ec556ab621014c7a07a6e844bff2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:17,077 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d030c0c1080b19b33ee54050ad4ea0a2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:17,493 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f 2023-02-09 22:55:17,493 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 2023-02-09 22:55:17,493 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f 2023-02-09 22:55:17,503 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/fcc50ba542b04c1cab2f19bc13ca18d6 2023-02-09 22:55:17,514 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead 2023-02-09 22:55:17,900 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/efe9f66995494ba78d74ee5225599fb6 2023-02-09 22:55:17,903 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/cac9ab38699a4e2fac99af68222a3f90 2023-02-09 22:55:17,918 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/26a11428bcba441ead1aecdbafb98061 2023-02-09 22:55:17,921 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/fcc50ba542b04c1cab2f19bc13ca18d6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6 2023-02-09 22:55:17,924 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:17,925 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/26a11428bcba441ead1aecdbafb98061 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061 2023-02-09 22:55:17,928 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:17,928 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 644cc6a94a1dc8fa8754303752b40531 in 852ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:17,928 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:17,928 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:17,928 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:18,316 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6 2023-02-09 22:55:18,317 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6 2023-02-09 22:55:18,323 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/13ff32351eca42509c4dbfadc1f72c98 2023-02-09 22:55:18,327 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/cac9ab38699a4e2fac99af68222a3f90 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90 2023-02-09 22:55:18,331 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:18,331 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/13ff32351eca42509c4dbfadc1f72c98 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98 2023-02-09 22:55:18,335 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:18,335 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d030c0c1080b19b33ee54050ad4ea0a2 in 1258ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:18,335 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:18,336 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:18,336 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:18,723 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/45b2586e54a4454b8ea853e99eb3525a 2023-02-09 22:55:18,727 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/efe9f66995494ba78d74ee5225599fb6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6 2023-02-09 22:55:18,730 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:18,730 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/45b2586e54a4454b8ea853e99eb3525a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a 2023-02-09 22:55:18,734 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:18,735 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1bc7ec556ab621014c7a07a6e844bff2 in 1658ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:18,735 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:18,735 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 22:55:18,735 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 22:55:18,737 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,737 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:18,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:18,737 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:18,737 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:18,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:18,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:18,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 22:55:18,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16278f7a[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:18,739 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 22:55:18,739 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,740 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,741 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 22:55:18,741 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 22:55:18,741 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:18,741 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,741 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:18,742 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:18,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:18,742 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 22:55:18,742 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:18,742 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 22:55:18,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:18,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:18,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:18,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:18,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 22:55:18,744 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 22:55:18,744 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:18,744 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:18,744 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 22:55:18,744 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:18,749 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,749 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:18,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:18,749 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:18,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,749 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:18,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,750 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:18,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:18,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:18,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 22:55:18,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:18,752 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:18,752 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:18,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:18,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:18,753 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:18,752 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,753 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:18,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:18,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:18,753 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,753 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:18,753 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,853 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:18,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:18,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:18,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:18,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:18,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:18,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:18,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 22:55:18,860 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 22:55:18,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:18,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 22:55:18,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 22:55:18,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,860 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:18,862 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:18,862 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:18,862 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:18,862 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:18,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:18,863 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 22:55:18,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 22:55:18,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 22:55:18,863 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 22:55:18,863 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:18,863 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 22:55:18,863 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:18,863 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:18,864 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:18,863 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:18,864 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:18,864 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:18,864 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:18,864 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:18,864 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d030c0c1080b19b33ee54050ad4ea0a2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:18,864 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 644cc6a94a1dc8fa8754303752b40531 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:18,864 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 1bc7ec556ab621014c7a07a6e844bff2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:19,279 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f 2023-02-09 22:55:19,279 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 2023-02-09 22:55:19,279 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb 2023-02-09 22:55:19,289 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/b4225150012c4e1f92b711ac1d5ccae2 2023-02-09 22:55:19,688 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/19c40713542541c0b2676be0471061d2 2023-02-09 22:55:19,689 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/dae3e28db95e402782a4642a708b9b8f 2023-02-09 22:55:19,701 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24 2023-02-09 22:55:20,103 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f 2023-02-09 22:55:20,103 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6 2023-02-09 22:55:20,108 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/c49570f4da0647cfae6e795d461a515d 2023-02-09 22:55:20,112 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/b4225150012c4e1f92b711ac1d5ccae2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2 2023-02-09 22:55:20,116 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:20,117 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/c49570f4da0647cfae6e795d461a515d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d 2023-02-09 22:55:20,120 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:20,121 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d030c0c1080b19b33ee54050ad4ea0a2 in 1257ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:20,121 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:20,121 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:20,121 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:20,508 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/d84bf35b04104c36b3a0c293372631c6 2023-02-09 22:55:20,508 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/ffdb38aa331c4ba9ab6cbceef95237ad 2023-02-09 22:55:20,511 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/dae3e28db95e402782a4642a708b9b8f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f 2023-02-09 22:55:20,512 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/19c40713542541c0b2676be0471061d2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2 2023-02-09 22:55:20,515 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:20,515 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:20,516 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/ffdb38aa331c4ba9ab6cbceef95237ad as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad 2023-02-09 22:55:20,516 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/d84bf35b04104c36b3a0c293372631c6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6 2023-02-09 22:55:20,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:20,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:20,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1bc7ec556ab621014c7a07a6e844bff2 in 1656ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:20,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 644cc6a94a1dc8fa8754303752b40531 in 1656ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:20,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:20,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:20,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:20,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 22:55:20,521 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 22:55:20,527 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,527 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:20,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:20,527 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,527 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:20,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 22:55:20,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64a7f02e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:20,529 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 22:55:20,529 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,529 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,530 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 22:55:20,530 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 22:55:20,530 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,530 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:20,530 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:20,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 22:55:20,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:20,531 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 22:55:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:20,533 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 22:55:20,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:20,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 22:55:20,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 22:55:20,533 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:20,534 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,534 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:20,534 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:20,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:20,535 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,536 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,538 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,538 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:20,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:20,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 22:55:20,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:20,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:20,539 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:20,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:20,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:20,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:20,639 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:20,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:20,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1580): Client=jenkins//172.31.2.10 snapshot request for:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:20,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675983320650 (current time:1675983320650). 2023-02-09 22:55:20,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 22:55:20,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] zookeeper.ReadOnlyZKClient(139): Connect 0x648298b4 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:20,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9452d45, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:20,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:20,663 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:37252, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:20,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x648298b4 to 127.0.0.1:59035 2023-02-09 22:55:20,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:20,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 22:55:20,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:20,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:20,673 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:20,674 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675983920674 2023-02-09 22:55:20,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:20,674 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1675983320640 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 22:55:20,675 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:20,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 22:55:20,676 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-09 22:55:20,676 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 22:55:20,677 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1675983920677 2023-02-09 22:55:20,679 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:20,679 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 22:55:20,779 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:20,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:20,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:20,783 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 22:55:20,983 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:20,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:20,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:20,984 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 22:55:21,083 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1675983320640 2023-02-09 22:55:21,084 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1675983320640' 2023-02-09 22:55:21,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 22:55:21,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1675983320640' starting 'acquire' 2023-02-09 22:55:21,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1675983320640', kicking off acquire phase on members. 2023-02-09 22:55:21,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1675983320640 2023-02-09 22:55:21,085 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,085 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 22:55:21,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 22:55:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:55:21,085 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,085 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1675983320640 2023-02-09 22:55:21,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-09 22:55:21,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1675983320640 2023-02-09 22:55:21,086 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1675983320640 from table testCompactionWithHFileLink type FLUSH 2023-02-09 22:55:21,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1675983320640 2023-02-09 22:55:21,088 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1675983320640' with timeout 300000ms 2023-02-09 22:55:21,088 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-09 22:55:21,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1675983320640' starting 'acquire' stage 2023-02-09 22:55:21,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1675983320640' locally acquired 2023-02-09 22:55:21,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (snaptb-1675983320640) in zk 2023-02-09 22:55:21,090 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,090 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:21,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 22:55:21,091 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,091 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1675983320640' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:21,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'snaptb-1675983320640' on coordinator 2023-02-09 22:55:21,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@82de2ed[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:21,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1675983320640' starting 'in-barrier' execution. 2023-02-09 22:55:21,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,093 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,093 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1675983320640' received 'reached' from coordinator. 2023-02-09 22:55:21,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:21,094 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:21,094 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:21,094 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:21,095 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.' region-info for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,095 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.' region-info for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,095 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.' region-info for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,100 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 22:55:21,100 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 22:55:21,100 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2] hfiles 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2] hfiles 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f] hfiles 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,103 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,106 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,106 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,106 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,106 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,106 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad] hfiles 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d] hfiles 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6] hfiles 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6 for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,107 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,284 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:21,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:21,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:21,285 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 22:55:21,513 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. completed. 2023-02-09 22:55:21,513 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. completed. 2023-02-09 22:55:21,513 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:21,513 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:21,513 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. completed. 2023-02-09 22:55:21,514 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1675983320640' locally completed 2023-02-09 22:55:21,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1675983320640' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:21,516 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1675983320640' has notified controller of completion 2023-02-09 22:55:21,516 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,516 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 22:55:21,516 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1675983320640' completed. 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1675983320640' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:21,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'snaptb-1675983320640', counting down latch. Waiting for 0 more 2023-02-09 22:55:21,517 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1675983320640' execution completed 2023-02-09 22:55:21,518 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:21,518 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:21,518 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1675983320640 2023-02-09 22:55:21,518 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1675983320640including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-09 22:55:21,519 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,519 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,519 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-09 22:55:21,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:55:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:21,519 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,520 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675983320640 2023-02-09 22:55:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:21,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1675983320640 because node does not exist (not an error) 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-09 22:55:21,524 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675983320640/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:21,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-09 22:55:21,524 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-09 22:55:21,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675983320640 2023-02-09 22:55:21,524 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675983320640 2023-02-09 22:55:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-09 22:55:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 22:55:21,525 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5] mob files 2023-02-09 22:55:21,525 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f 2023-02-09 22:55:21,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6] mob files 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6 2023-02-09 22:55:21,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6 2023-02-09 22:55:21,785 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:21,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:21,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:21,786 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 22:55:21,932 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1675983320640 2023-02-09 22:55:21,934 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/snaptb-1675983320640 2023-02-09 22:55:22,373 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/snaptb-1675983320640 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/snaptb-1675983320640 2023-02-09 22:55:22,786 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:22,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:22,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:22,787 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-09 22:55:22,791 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1675983320640 of table testCompactionWithHFileLink completed 2023-02-09 22:55:22,791 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/snaptb-1675983320640 2023-02-09 22:55:22,791 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/snaptb-1675983320640 2023-02-09 22:55:22,791 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1675983320640 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675983320674Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1675983321932 (+1258 ms)Consolidate snapshot: snaptb-1675983320640 at 1675983321932Loading Region manifests for snaptb-1675983320640 at 1675983321932Writing data manifest for snaptb-1675983320640 at 1675983321941 (+9 ms)Verifying snapshot: snaptb-1675983320640 at 1675983322347 (+406 ms)Snapshot snaptb-1675983320640 of table testCompactionWithHFileLink completed at 1675983322791 (+444 ms) 2023-02-09 22:55:22,792 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-09 22:55:22,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.1170 sec 2023-02-09 22:55:24,788 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:24,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:24,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 22:55:24,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:24,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:24,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-09 22:55:24,818 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-09 22:55:24,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:24,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-09 22:55:24,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-09 22:55:24,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:24,818 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:24,820 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:24,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:24,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:24,820 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:24,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:24,820 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:24,820 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:24,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:24,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:24,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:24,821 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-09 22:55:24,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-09 22:55:24,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-09 22:55:24,821 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-09 22:55:24,821 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:24,821 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-09 22:55:24,822 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:24,822 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:24,822 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:24,822 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:24,823 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:24,823 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. started... 2023-02-09 22:55:24,822 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. started... 2023-02-09 22:55:24,822 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. started... 2023-02-09 22:55:24,823 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing d030c0c1080b19b33ee54050ad4ea0a2 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:24,823 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 1bc7ec556ab621014c7a07a6e844bff2 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:24,823 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 644cc6a94a1dc8fa8754303752b40531 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:25,242 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/aafc8ac7946a463794d5d6f23ab130d0 2023-02-09 22:55:25,242 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/2baf79cffdf345ce992b2d2c8731c464 2023-02-09 22:55:25,242 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/c738092378454729bff193d56e1b5e23 2023-02-09 22:55:25,245 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aafc8ac7946a463794d5d6f23ab130d0 2023-02-09 22:55:25,245 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf79cffdf345ce992b2d2c8731c464 2023-02-09 22:55:25,246 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c738092378454729bff193d56e1b5e23 2023-02-09 22:55:26,061 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/6c47bc63213b4f65ad7b82fc1458dee6 2023-02-09 22:55:26,061 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/6cb9daac5d9242db89bfa2d3aed94791 2023-02-09 22:55:26,064 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/359d2b809b6444e3ab2979ef740c9724 2023-02-09 22:55:26,065 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cb9daac5d9242db89bfa2d3aed94791 2023-02-09 22:55:26,065 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c47bc63213b4f65ad7b82fc1458dee6 2023-02-09 22:55:26,065 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/c738092378454729bff193d56e1b5e23 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23 2023-02-09 22:55:26,066 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/aafc8ac7946a463794d5d6f23ab130d0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0 2023-02-09 22:55:26,068 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 359d2b809b6444e3ab2979ef740c9724 2023-02-09 22:55:26,069 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/2baf79cffdf345ce992b2d2c8731c464 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464 2023-02-09 22:55:26,070 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aafc8ac7946a463794d5d6f23ab130d0 2023-02-09 22:55:26,070 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,071 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c738092378454729bff193d56e1b5e23 2023-02-09 22:55:26,071 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,071 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/6cb9daac5d9242db89bfa2d3aed94791 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791 2023-02-09 22:55:26,072 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/6c47bc63213b4f65ad7b82fc1458dee6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6 2023-02-09 22:55:26,074 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2baf79cffdf345ce992b2d2c8731c464 2023-02-09 22:55:26,074 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464, entries=3, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,075 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/359d2b809b6444e3ab2979ef740c9724 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724 2023-02-09 22:55:26,076 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6cb9daac5d9242db89bfa2d3aed94791 2023-02-09 22:55:26,076 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,076 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c47bc63213b4f65ad7b82fc1458dee6 2023-02-09 22:55:26,076 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,077 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 644cc6a94a1dc8fa8754303752b40531 in 1254ms, sequenceid=24, compaction requested=true 2023-02-09 22:55:26,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:26,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:26,077 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d030c0c1080b19b33ee54050ad4ea0a2 in 1254ms, sequenceid=24, compaction requested=true 2023-02-09 22:55:26,077 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:26,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:26,077 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:26,077 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:26,080 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 359d2b809b6444e3ab2979ef740c9724 2023-02-09 22:55:26,081 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724, entries=1, sequenceid=24, filesize=5.0 K 2023-02-09 22:55:26,081 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1bc7ec556ab621014c7a07a6e844bff2 in 1258ms, sequenceid=24, compaction requested=true 2023-02-09 22:55:26,081 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:26,081 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:26,081 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:26,081 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:26,081 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,082 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-09 22:55:26,082 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-09 22:55:26,083 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:26,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:26,084 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,084 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:26,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:26,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:26,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:26,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-09 22:55:26,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d20ab88[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:26,085 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-09 22:55:26,085 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,086 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-09 22:55:26,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-09 22:55:26,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:26,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:26,091 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,091 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-09 22:55:26,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:26,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:26,091 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:26,091 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-09 22:55:26,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:26,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:26,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:26,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:26,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-09 22:55:26,093 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-09 22:55:26,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:26,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:26,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-09 22:55:26,093 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:26,095 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,095 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:26,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,095 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:26,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:26,096 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,097 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:26,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-09 22:55:26,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:26,099 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-09 22:55:26,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-09 22:55:26,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:26,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:26,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-09 22:55:26,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:26,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:26,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:26,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-09 22:55:26,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-09 22:55:26,200 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:26,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:26,201 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:26,201 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 644cc6a94a1dc8fa8754303752b40531/family1 is initiating major compaction (all files) 2023-02-09 22:55:26,201 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:26,202 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:26,202 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 644cc6a94a1dc8fa8754303752b40531/family1 in testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:26,202 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp, totalSize=26.7 K 2023-02-09 22:55:26,203 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 4e7a9faf25f04cfea21bb52c9ed5615a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:26,203 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 6caab206f3eb4f499669f0c3f437a6b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:26,204 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting fcc50ba542b04c1cab2f19bc13ca18d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:26,204 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 19c40713542541c0b2676be0471061d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:26,205 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting aafc8ac7946a463794d5d6f23ab130d0, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:26,217 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 644cc6a94a1dc8fa8754303752b40531#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:26,626 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del 2023-02-09 22:55:26,626 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del 2023-02-09 22:55:27,438 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02b7704fde3475698f9c7068823c825 2023-02-09 22:55:27,439 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/d02b7704fde3475698f9c7068823c825 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/d02b7704fde3475698f9c7068823c825 2023-02-09 22:55:27,442 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d02b7704fde3475698f9c7068823c825 2023-02-09 22:55:27,443 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 644cc6a94a1dc8fa8754303752b40531/family1 of 644cc6a94a1dc8fa8754303752b40531 into d02b7704fde3475698f9c7068823c825(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:27,443 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:27,444 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:27,444 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 644cc6a94a1dc8fa8754303752b40531/family2 is initiating major compaction (all files) 2023-02-09 22:55:27,444 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:27,444 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:27,444 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 644cc6a94a1dc8fa8754303752b40531/family2 in testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:27,445 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp, totalSize=25.4 K 2023-02-09 22:55:27,445 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 109596cc15d14cfea976ed9bedf2377f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:27,445 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 94ad938e948e4b0eb64085e2a5d251d4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:27,446 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 26a11428bcba441ead1aecdbafb98061, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:27,446 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting d84bf35b04104c36b3a0c293372631c6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:27,446 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 6cb9daac5d9242db89bfa2d3aed94791, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:27,457 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 644cc6a94a1dc8fa8754303752b40531#family2#compaction#109 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:27,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0] to archive 2023-02-09 22:55:27,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:27,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/4e7a9faf25f04cfea21bb52c9ed5615a 2023-02-09 22:55:27,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/6caab206f3eb4f499669f0c3f437a6b0 2023-02-09 22:55:27,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/fcc50ba542b04c1cab2f19bc13ca18d6 2023-02-09 22:55:27,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/19c40713542541c0b2676be0471061d2 2023-02-09 22:55:27,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/aafc8ac7946a463794d5d6f23ab130d0 2023-02-09 22:55:27,865 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del 2023-02-09 22:55:27,865 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del 2023-02-09 22:55:28,678 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f2578521deb42ce9973104ff8e9d64b 2023-02-09 22:55:28,679 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/5f2578521deb42ce9973104ff8e9d64b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/5f2578521deb42ce9973104ff8e9d64b 2023-02-09 22:55:28,682 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f2578521deb42ce9973104ff8e9d64b 2023-02-09 22:55:28,683 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 644cc6a94a1dc8fa8754303752b40531/family2 of 644cc6a94a1dc8fa8754303752b40531 into 5f2578521deb42ce9973104ff8e9d64b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:28,683 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:28,683 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:28,683 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 1bc7ec556ab621014c7a07a6e844bff2/family1 is initiating major compaction (all files) 2023-02-09 22:55:28,683 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:28,683 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:28,684 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 1bc7ec556ab621014c7a07a6e844bff2/family1 in testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:28,684 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp, totalSize=26.7 K 2023-02-09 22:55:28,684 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting c85ef1364b9e4b06abc3c6c00f1d96ee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313487 2023-02-09 22:55:28,685 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting cd51f7493e8d4aa19d1ec014571098c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:28,685 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting efe9f66995494ba78d74ee5225599fb6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:28,685 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting dae3e28db95e402782a4642a708b9b8f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:28,686 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 2baf79cffdf345ce992b2d2c8731c464, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:28,696 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 1bc7ec556ab621014c7a07a6e844bff2#family1#compaction#110 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:28,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791] to archive 2023-02-09 22:55:28,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:28,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/109596cc15d14cfea976ed9bedf2377f 2023-02-09 22:55:28,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/94ad938e948e4b0eb64085e2a5d251d4 2023-02-09 22:55:28,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/26a11428bcba441ead1aecdbafb98061 2023-02-09 22:55:28,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/d84bf35b04104c36b3a0c293372631c6 2023-02-09 22:55:28,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/6cb9daac5d9242db89bfa2d3aed94791 2023-02-09 22:55:29,107 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del 2023-02-09 22:55:29,107 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del 2023-02-09 22:55:29,920 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 905783ad4411479c8b3491ac50214c9e 2023-02-09 22:55:29,921 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/905783ad4411479c8b3491ac50214c9e as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/905783ad4411479c8b3491ac50214c9e 2023-02-09 22:55:29,924 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 905783ad4411479c8b3491ac50214c9e 2023-02-09 22:55:29,925 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1bc7ec556ab621014c7a07a6e844bff2/family1 of 1bc7ec556ab621014c7a07a6e844bff2 into 905783ad4411479c8b3491ac50214c9e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:29,925 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:29,925 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:29,925 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 1bc7ec556ab621014c7a07a6e844bff2/family2 is initiating major compaction (all files) 2023-02-09 22:55:29,925 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:29,925 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:29,926 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 1bc7ec556ab621014c7a07a6e844bff2/family2 in testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:29,926 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp, totalSize=25.4 K 2023-02-09 22:55:29,926 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 711ccfffc8ca4f1f8a6d9a02c4efbcba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313487 2023-02-09 22:55:29,927 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 5dd6f138a437444facbaa6253289b6f5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:29,927 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 45b2586e54a4454b8ea853e99eb3525a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:29,927 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting ffdb38aa331c4ba9ab6cbceef95237ad, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:29,927 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 359d2b809b6444e3ab2979ef740c9724, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:29,936 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 1bc7ec556ab621014c7a07a6e844bff2#family2#compaction#111 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:29,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464] to archive 2023-02-09 22:55:29,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:29,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/c85ef1364b9e4b06abc3c6c00f1d96ee 2023-02-09 22:55:29,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/cd51f7493e8d4aa19d1ec014571098c0 2023-02-09 22:55:29,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/efe9f66995494ba78d74ee5225599fb6 2023-02-09 22:55:29,982 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/dae3e28db95e402782a4642a708b9b8f 2023-02-09 22:55:29,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/2baf79cffdf345ce992b2d2c8731c464 2023-02-09 22:55:30,344 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del 2023-02-09 22:55:30,344 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del 2023-02-09 22:55:31,157 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 339ea30fef3f415eacdbe48bce5191ff 2023-02-09 22:55:31,158 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/339ea30fef3f415eacdbe48bce5191ff as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/339ea30fef3f415eacdbe48bce5191ff 2023-02-09 22:55:31,161 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 339ea30fef3f415eacdbe48bce5191ff 2023-02-09 22:55:31,162 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1bc7ec556ab621014c7a07a6e844bff2/family2 of 1bc7ec556ab621014c7a07a6e844bff2 into 339ea30fef3f415eacdbe48bce5191ff(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:31,162 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:31,162 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:31,163 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): d030c0c1080b19b33ee54050ad4ea0a2/family1 is initiating major compaction (all files) 2023-02-09 22:55:31,163 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:31,163 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:31,163 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of d030c0c1080b19b33ee54050ad4ea0a2/family1 in testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:31,163 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp, totalSize=26.7 K 2023-02-09 22:55:31,164 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 55831fc6ce994a5cb81893af3b5f34fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:31,164 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 978c8825afba40b3a59aa1565689d9c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:31,165 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting cac9ab38699a4e2fac99af68222a3f90, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:31,165 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting b4225150012c4e1f92b711ac1d5ccae2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:31,165 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting c738092378454729bff193d56e1b5e23, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:31,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724] to archive 2023-02-09 22:55:31,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:31,175 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): d030c0c1080b19b33ee54050ad4ea0a2#family1#compaction#112 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:31,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/711ccfffc8ca4f1f8a6d9a02c4efbcba 2023-02-09 22:55:31,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/5dd6f138a437444facbaa6253289b6f5 2023-02-09 22:55:31,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/45b2586e54a4454b8ea853e99eb3525a 2023-02-09 22:55:31,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/ffdb38aa331c4ba9ab6cbceef95237ad 2023-02-09 22:55:31,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/359d2b809b6444e3ab2979ef740c9724 2023-02-09 22:55:31,583 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del 2023-02-09 22:55:31,583 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del 2023-02-09 22:55:32,397 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e74b12ad198d48a3b3bec3c77272775c 2023-02-09 22:55:32,397 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/e74b12ad198d48a3b3bec3c77272775c as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/e74b12ad198d48a3b3bec3c77272775c 2023-02-09 22:55:32,401 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e74b12ad198d48a3b3bec3c77272775c 2023-02-09 22:55:32,402 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d030c0c1080b19b33ee54050ad4ea0a2/family1 of d030c0c1080b19b33ee54050ad4ea0a2 into e74b12ad198d48a3b3bec3c77272775c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:32,402 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:32,402 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:32,402 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): d030c0c1080b19b33ee54050ad4ea0a2/family2 is initiating major compaction (all files) 2023-02-09 22:55:32,402 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:32,402 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:32,403 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of d030c0c1080b19b33ee54050ad4ea0a2/family2 in testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:32,403 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp, totalSize=25.4 K 2023-02-09 22:55:32,403 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting e90eb10934884f2592db3b80bd585a91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:32,404 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 74769bcc3f08490d9e45153b83bc91a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:32,404 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 13ff32351eca42509c4dbfadc1f72c98, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:32,404 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting c49570f4da0647cfae6e795d461a515d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:32,405 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 6c47bc63213b4f65ad7b82fc1458dee6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-09 22:55:32,416 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): d030c0c1080b19b33ee54050ad4ea0a2#family2#compaction#113 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:32,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23] to archive 2023-02-09 22:55:32,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:32,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/55831fc6ce994a5cb81893af3b5f34fd 2023-02-09 22:55:32,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/978c8825afba40b3a59aa1565689d9c0 2023-02-09 22:55:32,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cac9ab38699a4e2fac99af68222a3f90 2023-02-09 22:55:32,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/b4225150012c4e1f92b711ac1d5ccae2 2023-02-09 22:55:32,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/c738092378454729bff193d56e1b5e23 2023-02-09 22:55:32,824 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del 2023-02-09 22:55:32,825 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del 2023-02-09 22:55:33,638 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a013e9ca16f467b835875ab28ce70bc 2023-02-09 22:55:33,639 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/4a013e9ca16f467b835875ab28ce70bc as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/4a013e9ca16f467b835875ab28ce70bc 2023-02-09 22:55:33,642 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4a013e9ca16f467b835875ab28ce70bc 2023-02-09 22:55:33,643 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d030c0c1080b19b33ee54050ad4ea0a2/family2 of d030c0c1080b19b33ee54050ad4ea0a2 into 4a013e9ca16f467b835875ab28ce70bc(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:33,643 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:33,648 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:33,651 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:33,654 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:33,655 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:33,661 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:55:33,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6] to archive 2023-02-09 22:55:33,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:33,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/e90eb10934884f2592db3b80bd585a91 2023-02-09 22:55:33,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/74769bcc3f08490d9e45153b83bc91a7 2023-02-09 22:55:33,686 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/13ff32351eca42509c4dbfadc1f72c98 2023-02-09 22:55:33,687 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/c49570f4da0647cfae6e795d461a515d 2023-02-09 22:55:33,688 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/6c47bc63213b4f65ad7b82fc1458dee6 2023-02-09 22:55:33,691 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 22:55:33,694 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del 2023-02-09 22:55:33,697 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del 2023-02-09 22:55:33,700 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del 2023-02-09 22:55:33,700 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:33,700 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(139): Connect 0x07c7517f to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:33,705 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@180dfa3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:33,706 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:33,706 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:33,706 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:34,559 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349 2023-02-09 22:55:34,560 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,560 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a 2023-02-09 22:55:34,561 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,568 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519 2023-02-09 22:55:34,568 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,573 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:34,577 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:34926, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:34,582 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:34,584 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58228, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:34,598 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/92edacf2e55f448bb05e42b70319cf82 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 22:55:34,598 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/876971f976ca41369d643fa7a2767cc2 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 22:55:34,605 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/92edacf2e55f448bb05e42b70319cf82}] 2023-02-09 22:55:34,605 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/876971f976ca41369d643fa7a2767cc2}] 2023-02-09 22:55:34,610 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/eb7f456fe5f74980bbec7ceab3c927b0 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 22:55:34,611 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/eb7f456fe5f74980bbec7ceab3c927b0}] 2023-02-09 22:55:34,611 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/92edacf2e55f448bb05e42b70319cf82 for inclusion in 644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:34,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/876971f976ca41369d643fa7a2767cc2 for inclusion in 1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:34,613 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/eb7f456fe5f74980bbec7ceab3c927b0 for inclusion in d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:34,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 22:55:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 22:55:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 22:55:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 22:55:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:34,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/92edacf2e55f448bb05e42b70319cf82 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__ttgmg7hlkn40r0lsk9sk7inu44khncuin9ap4l2u3f0obs13f35ih0709cv2qu73/family1/92edacf2e55f448bb05e42b70319cf82 2023-02-09 22:55:34,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/876971f976ca41369d643fa7a2767cc2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__astn12m474eftb2vu2vhpqe75ad4magq4nb00u44taljb0ik9105ohu47bi7ru7d/family1/876971f976ca41369d643fa7a2767cc2 2023-02-09 22:55:34,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 22:55:34,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 22:55:34,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:34,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/eb7f456fe5f74980bbec7ceab3c927b0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__u4jicu20r6psnec3ddj03c6dp29q2va4hl82ed6ur0d7igjn50tl3i68uet2f0oj/family1/eb7f456fe5f74980bbec7ceab3c927b0 2023-02-09 22:55:34,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__ttgmg7hlkn40r0lsk9sk7inu44khncuin9ap4l2u3f0obs13f35ih0709cv2qu73/family1/92edacf2e55f448bb05e42b70319cf82 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ 2023-02-09 22:55:34,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__astn12m474eftb2vu2vhpqe75ad4magq4nb00u44taljb0ik9105ohu47bi7ru7d/family1/876971f976ca41369d643fa7a2767cc2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ 2023-02-09 22:55:34,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__ttgmg7hlkn40r0lsk9sk7inu44khncuin9ap4l2u3f0obs13f35ih0709cv2qu73/family1/92edacf2e55f448bb05e42b70319cf82 into 644cc6a94a1dc8fa8754303752b40531/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ - updating store file list. 2023-02-09 22:55:34,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__astn12m474eftb2vu2vhpqe75ad4magq4nb00u44taljb0ik9105ohu47bi7ru7d/family1/876971f976ca41369d643fa7a2767cc2 into 1bc7ec556ab621014c7a07a6e844bff2/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ - updating store file list. 2023-02-09 22:55:34,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__u4jicu20r6psnec3ddj03c6dp29q2va4hl82ed6ur0d7igjn50tl3i68uet2f0oj/family1/eb7f456fe5f74980bbec7ceab3c927b0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ 2023-02-09 22:55:34,618 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__u4jicu20r6psnec3ddj03c6dp29q2va4hl82ed6ur0d7igjn50tl3i68uet2f0oj/family1/eb7f456fe5f74980bbec7ceab3c927b0 into d030c0c1080b19b33ee54050ad4ea0a2/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ - updating store file list. 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ into 1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ into 644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__astn12m474eftb2vu2vhpqe75ad4magq4nb00u44taljb0ik9105ohu47bi7ru7d/family1/876971f976ca41369d643fa7a2767cc2 into 1bc7ec556ab621014c7a07a6e844bff2/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_) 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__ttgmg7hlkn40r0lsk9sk7inu44khncuin9ap4l2u3f0obs13f35ih0709cv2qu73/family1/92edacf2e55f448bb05e42b70319cf82 into 644cc6a94a1dc8fa8754303752b40531/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_) 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ into d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__u4jicu20r6psnec3ddj03c6dp29q2va4hl82ed6ur0d7igjn50tl3i68uet2f0oj/family1/eb7f456fe5f74980bbec7ceab3c927b0 into d030c0c1080b19b33ee54050ad4ea0a2/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_) 2023-02-09 22:55:34,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__astn12m474eftb2vu2vhpqe75ad4magq4nb00u44taljb0ik9105ohu47bi7ru7d/family1/876971f976ca41369d643fa7a2767cc2 2023-02-09 22:55:34,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__ttgmg7hlkn40r0lsk9sk7inu44khncuin9ap4l2u3f0obs13f35ih0709cv2qu73/family1/92edacf2e55f448bb05e42b70319cf82 2023-02-09 22:55:34,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__u4jicu20r6psnec3ddj03c6dp29q2va4hl82ed6ur0d7igjn50tl3i68uet2f0oj/family1/eb7f456fe5f74980bbec7ceab3c927b0 2023-02-09 22:55:34,625 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,625 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,625 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,625 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,625 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,626 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,627 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb 2023-02-09 22:55:34,628 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 2023-02-09 22:55:34,628 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 2023-02-09 22:55:34,628 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 2023-02-09 22:55:34,629 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b 2023-02-09 22:55:34,629 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f 2023-02-09 22:55:34,630 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 2023-02-09 22:55:34,630 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f 2023-02-09 22:55:34,630 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc 2023-02-09 22:55:34,631 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f 2023-02-09 22:55:34,631 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:34,631 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 2023-02-09 22:55:34,631 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 22:55:34,631 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:34,631 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 22:55:34,632 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 2023-02-09 22:55:34,632 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:34,632 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 22:55:34,632 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:34,632 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:34,632 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:34,632 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:34,632 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:34,632 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del] 2023-02-09 22:55:34,633 DEBUG [Listener at localhost/33819] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,634 DEBUG [Listener at localhost/33819] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020951e6adcaf1f349fda9a77b3bf68d54af_del 2023-02-09 22:55:34,634 INFO [Listener at localhost/33819] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 22:55:34,634 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del] 2023-02-09 22:55:34,635 DEBUG [Listener at localhost/33819] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,636 DEBUG [Listener at localhost/33819] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209cb764a0f38ec4fb98e50ff9ab1731177_del 2023-02-09 22:55:34,636 INFO [Listener at localhost/33819] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 22:55:34,636 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del] 2023-02-09 22:55:34,636 DEBUG [Listener at localhost/33819] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:34,637 DEBUG [Listener at localhost/33819] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209b285ebf066ae41f2990bb36e8f8c7ccb_del 2023-02-09 22:55:34,637 INFO [Listener at localhost/33819] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-09 22:55:34,640 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:34,643 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:34,646 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:34,646 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:34,652 INFO [Listener at localhost/33819] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-09 22:55:34,652 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.HMaster$11(2405): Client=jenkins//172.31.2.10 disable testCompactionWithHFileLink 2023-02-09 22:55:34,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-09 22:55:34,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:34,656 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983334656"}]},"ts":"1675983334656"} 2023-02-09 22:55:34,657 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-09 22:55:34,658 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-09 22:55:34,659 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, UNASSIGN}] 2023-02-09 22:55:34,661 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, UNASSIGN 2023-02-09 22:55:34,661 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, UNASSIGN 2023-02-09 22:55:34,661 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, UNASSIGN 2023-02-09 22:55:34,661 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:34,661 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:34,661 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=CLOSING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:34,661 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983334661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983334661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983334661"}]},"ts":"1675983334661"} 2023-02-09 22:55:34,662 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983334661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983334661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983334661"}]},"ts":"1675983334661"} 2023-02-09 22:55:34,662 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983334661"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983334661"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983334661"}]},"ts":"1675983334661"} 2023-02-09 22:55:34,663 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:34,663 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:34,664 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:34,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:34,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:34,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 644cc6a94a1dc8fa8754303752b40531, disabling compactions & flushes 2023-02-09 22:55:34,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:34,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:34,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. after waiting 0 ms 2023-02-09 22:55:34,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:34,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 22:55:34,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:34,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 644cc6a94a1dc8fa8754303752b40531: Waiting for close lock at 1675983334815Running coprocessor pre-close hooks at 1675983334815Disabling compacts and flushes for region at 1675983334815Disabling writes for close at 1675983334815Writing region close event to WAL at 1675983334816 (+1 ms)Running coprocessor post-close hooks at 1675983334819 (+3 ms)Closed at 1675983334820 (+1 ms) 2023-02-09 22:55:34,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:34,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:34,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing d030c0c1080b19b33ee54050ad4ea0a2, disabling compactions & flushes 2023-02-09 22:55:34,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:34,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:34,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. after waiting 0 ms 2023-02-09 22:55:34,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:34,822 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=CLOSED 2023-02-09 22:55:34,822 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983334822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983334822"}]},"ts":"1675983334822"} 2023-02-09 22:55:34,824 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-02-09 22:55:34,824 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026 in 159 msec 2023-02-09 22:55:34,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 22:55:34,825 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, UNASSIGN in 165 msec 2023-02-09 22:55:34,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:34,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for d030c0c1080b19b33ee54050ad4ea0a2: Waiting for close lock at 1675983334822Running coprocessor pre-close hooks at 1675983334822Disabling compacts and flushes for region at 1675983334822Disabling writes for close at 1675983334822Writing region close event to WAL at 1675983334823 (+1 ms)Running coprocessor post-close hooks at 1675983334826 (+3 ms)Closed at 1675983334826 2023-02-09 22:55:34,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:34,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(109): Close 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:34,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 1bc7ec556ab621014c7a07a6e844bff2, disabling compactions & flushes 2023-02-09 22:55:34,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:34,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:34,828 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=CLOSED 2023-02-09 22:55:34,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. after waiting 0 ms 2023-02-09 22:55:34,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:34,828 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983334828"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983334828"}]},"ts":"1675983334828"} 2023-02-09 22:55:34,830 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 22:55:34,830 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 166 msec 2023-02-09 22:55:34,831 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, UNASSIGN in 171 msec 2023-02-09 22:55:34,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-09 22:55:34,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:34,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 1bc7ec556ab621014c7a07a6e844bff2: Waiting for close lock at 1675983334828Running coprocessor pre-close hooks at 1675983334828Disabling compacts and flushes for region at 1675983334828Disabling writes for close at 1675983334828Writing region close event to WAL at 1675983334829 (+1 ms)Running coprocessor post-close hooks at 1675983334832 (+3 ms)Closed at 1675983334832 2023-02-09 22:55:34,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.UnassignRegionHandler(147): Closed 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:34,834 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=CLOSED 2023-02-09 22:55:34,834 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983334834"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983334834"}]},"ts":"1675983334834"} 2023-02-09 22:55:34,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-02-09 22:55:34,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 172 msec 2023-02-09 22:55:34,837 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-09 22:55:34,837 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, UNASSIGN in 177 msec 2023-02-09 22:55:34,837 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983334837"}]},"ts":"1675983334837"} 2023-02-09 22:55:34,838 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-09 22:55:34,839 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-09 22:55:34,841 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 188 msec 2023-02-09 22:55:34,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 22:55:34,958 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-09 22:55:35,112 INFO [regionserver/jenkins-hbase9:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-09 22:55:35,117 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:55:35,117 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x14035645 to 127.0.0.1:59035 2023-02-09 22:55:35,117 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,119 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:55:35,119 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fc4a5f1 to 127.0.0.1:59035 2023-02-09 22:55:35,119 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,119 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bb8c256 to 127.0.0.1:59035 2023-02-09 22:55:35,119 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,122 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:55:35,122 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26bd9038 to 127.0.0.1:59035 2023-02-09 22:55:35,122 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,122 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:55:35,122 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x07c7517f to 127.0.0.1:59035 2023-02-09 22:55:35,122 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,127 INFO [Listener at localhost/33819] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:35,128 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1580): Client=jenkins//172.31.2.10 snapshot request for:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:35,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675983335128 (current time:1675983335128). 2023-02-09 22:55:35,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-09 22:55:35,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] zookeeper.ReadOnlyZKClient(139): Connect 0x53f38fd3 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:35,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b1985dc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:35,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:35,136 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:34928, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:35,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53f38fd3 to 127.0.0.1:59035 2023-02-09 22:55:35,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:55:35,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-09 22:55:35,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:35,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:35,140 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:35,140 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675983935140 2023-02-09 22:55:35,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:35,140 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1675983320640-1675983335127 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-09 22:55:35,141 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:35,141 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-09 22:55:35,241 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:35,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:35,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:35,242 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-09 22:55:35,442 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:35,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:35,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:35,443 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-09 22:55:35,547 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 22:55:35,549 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/d02b7704fde3475698f9c7068823c825] hfiles 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/d02b7704fde3475698f9c7068823c825 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/e74b12ad198d48a3b3bec3c77272775c] hfiles 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a] hfiles 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ 2023-02-09 22:55:35,552 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/905783ad4411479c8b3491ac50214c9e] hfiles 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/e74b12ad198d48a3b3bec3c77272775c 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/905783ad4411479c8b3491ac50214c9e 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349 2023-02-09 22:55:35,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/5f2578521deb42ce9973104ff8e9d64b] hfiles 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/4a013e9ca16f467b835875ab28ce70bc] hfiles 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/4a013e9ca16f467b835875ab28ce70bc 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/5f2578521deb42ce9973104ff8e9d64b 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/339ea30fef3f415eacdbe48bce5191ff] hfiles 2023-02-09 22:55:35,554 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/339ea30fef3f415eacdbe48bce5191ff 2023-02-09 22:55:35,559 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6] hfiles 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302092d603463edc244b7809c4c5b46d14ba6 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209805f38d77a76461aa01597acea32ab3a 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020987317be797d4430c8990d55cb95ec44d 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230209ef652dc1a4314cd781bfecfc0a118c24 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093dae665e464c41a6ab4412246e0c159f 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302093e0f7dfef44f4c039bf2156f1977fead 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209cd8b29a582b843c6bca5cb3fe9d424e0 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230209f71864ff56a8464b9ee5aa33a0003ab6 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020910a7c635ab574675acfe6f10605c3491 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302095d13e0756f154bf3b7c37eb965ff3a6a 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020975e18c421434449c840e555c49414ea6 2023-02-09 22:55:35,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230209e37d3f05bf194bed939727f316d4acc6 2023-02-09 22:55:35,743 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:35,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:35,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:35,744 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-09 22:55:35,969 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-09 22:55:35,969 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:35,970 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:35,990 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675983320640-1675983335127 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:36,244 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:36,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:36,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-09 22:55:36,246 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-09 22:55:36,804 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1675983320640-1675983335127 of table testCompactionWithHFileLink completed 2023-02-09 22:55:36,804 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:36,805 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:36,805 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase9:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1675983320640-1675983335127 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675983335140Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1675983335547 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1675983320640-1675983335127 at 1675983335969 (+422 ms)Loading Region manifests for hbase-failsafe-snaptb-1675983320640-1675983335127 at 1675983335969Writing data manifest for hbase-failsafe-snaptb-1675983320640-1675983335127 at 1675983335972 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1675983320640-1675983335127 at 1675983335977 (+5 ms)Snapshot hbase-failsafe-snaptb-1675983320640-1675983335127 of table testCompactionWithHFileLink completed at 1675983336804 (+827 ms) 2023-02-09 22:55:36,806 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-09 22:55:36,807 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.6680 sec 2023-02-09 22:55:36,965 DEBUG [regionserver/jenkins-hbase9:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 22:55:37,246 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-09 22:55:37,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-09 22:55:37,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1675983320640-1675983335127 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-09 22:55:37,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675983320640" table: "testCompactionWithHFileLink" creation_time: 1675983320650 type: FLUSH version: 2 ttl: 0 ) 2023-02-09 22:55:37,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1675983320640 as table=testCompactionWithHFileLink 2023-02-09 22:55:37,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:37,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:37,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:37,701 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 22:55:37,701 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:37,701 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-09 22:55:37,702 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-09 22:55:37,706 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1675983320640" table: "testCompactionWithHFileLink" creation_time: 1675983320650 type: FLUSH version: 2 ttl: 0 2023-02-09 22:55:37,707 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink 2023-02-09 22:55:37,712 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-09 22:55:37,716 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230209d0d317f95167492dbf3e158e37523519 2023-02-09 22:55:37,717 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209b2c5132946fc46ff9191403e69973b1a 2023-02-09 22:55:37,718 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209568671041d39471d98acc4131a0cf349 2023-02-09 22:55:37,718 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,721 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,724 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,726 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,728 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,731 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,733 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,736 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,739 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,742 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,745 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,748 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 of snapshot=snaptb-1675983320640 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-09 22:55:37,753 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020998564af35d54465cb1eafc2b49aa951e_del 2023-02-09 22:55:37,753 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302096f7f074be89c4c35ae3feb1e52c62a7c_del 2023-02-09 22:55:37,755 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302090d147a2e6cb247a78b58deb11baf0be5_del 2023-02-09 22:55:37,755 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:37,755 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:37,755 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:37,758 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/d02b7704fde3475698f9c7068823c825 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/d02b7704fde3475698f9c7068823c825 2023-02-09 22:55:37,758 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/cd9c287ac456462d8d96191c14e85554_SeqId_29_ 2023-02-09 22:55:37,758 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/1dc32e8ff30f4e6c91d3f3c6e5bbea84_SeqId_29_ 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/9fac186e439340a39dc1a8055e872954_SeqId_29_ 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4e7a9faf25f04cfea21bb52c9ed5615a of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/e74b12ad198d48a3b3bec3c77272775c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/e74b12ad198d48a3b3bec3c77272775c 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55831fc6ce994a5cb81893af3b5f34fd of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/905783ad4411479c8b3491ac50214c9e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/905783ad4411479c8b3491ac50214c9e 2023-02-09 22:55:37,760 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c85ef1364b9e4b06abc3c6c00f1d96ee of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,764 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6caab206f3eb4f499669f0c3f437a6b0 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,764 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 978c8825afba40b3a59aa1565689d9c0 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,764 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cd51f7493e8d4aa19d1ec014571098c0 of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,768 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cac9ab38699a4e2fac99af68222a3f90 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,768 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fcc50ba542b04c1cab2f19bc13ca18d6 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,768 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink efe9f66995494ba78d74ee5225599fb6 of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,771 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b4225150012c4e1f92b711ac1d5ccae2 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,771 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 19c40713542541c0b2676be0471061d2 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,771 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dae3e28db95e402782a4642a708b9b8f of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,784 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/4a013e9ca16f467b835875ab28ce70bc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/4a013e9ca16f467b835875ab28ce70bc 2023-02-09 22:55:37,785 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e90eb10934884f2592db3b80bd585a91 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,785 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/5f2578521deb42ce9973104ff8e9d64b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/5f2578521deb42ce9973104ff8e9d64b 2023-02-09 22:55:37,785 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 109596cc15d14cfea976ed9bedf2377f of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,785 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/339ea30fef3f415eacdbe48bce5191ff to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/339ea30fef3f415eacdbe48bce5191ff 2023-02-09 22:55:37,785 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 711ccfffc8ca4f1f8a6d9a02c4efbcba of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,792 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 74769bcc3f08490d9e45153b83bc91a7 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,792 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 94ad938e948e4b0eb64085e2a5d251d4 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,793 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5dd6f138a437444facbaa6253289b6f5 of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,796 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 13ff32351eca42509c4dbfadc1f72c98 of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,796 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 26a11428bcba441ead1aecdbafb98061 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,797 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 45b2586e54a4454b8ea853e99eb3525a of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,801 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c49570f4da0647cfae6e795d461a515d of snapshot=snaptb-1675983320640 to region=d030c0c1080b19b33ee54050ad4ea0a2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,801 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d84bf35b04104c36b3a0c293372631c6 of snapshot=snaptb-1675983320640 to region=644cc6a94a1dc8fa8754303752b40531 table=testCompactionWithHFileLink 2023-02-09 22:55:37,802 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ffdb38aa331c4ba9ab6cbceef95237ad of snapshot=snaptb-1675983320640 to region=1bc7ec556ab621014c7a07a6e844bff2 table=testCompactionWithHFileLink 2023-02-09 22:55:37,805 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1675983320640" table: "testCompactionWithHFileLink" creation_time: 1675983320650 type: FLUSH version: 2 ttl: 0 2023-02-09 22:55:37,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983337806"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:37,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983337806"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:37,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675983337806"}]},"ts":"9223372036854775807"} 2023-02-09 22:55:37,808 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 22:55:37,808 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1bc7ec556ab621014c7a07a6e844bff2, NAME => 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 644cc6a94a1dc8fa8754303752b40531, NAME => 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => d030c0c1080b19b33ee54050ad4ea0a2, NAME => 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 22:55:37,808 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983337807"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983337807"}]},"ts":"1675983337807"} 2023-02-09 22:55:37,808 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983337807"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983337807"}]},"ts":"1675983337807"} 2023-02-09 22:55:37,808 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983337807"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983337807"}]},"ts":"1675983337807"} 2023-02-09 22:55:37,809 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 22:55:37,809 INFO [PEWorker-1] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-09 22:55:37,810 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 1bc7ec556ab621014c7a07a6e844bff2, NAME => 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 644cc6a94a1dc8fa8754303752b40531, NAME => 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => d030c0c1080b19b33ee54050ad4ea0a2, NAME => 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-09 22:55:37,810 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1675983320640 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-09 22:55:37,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675983320640" table: "testCompactionWithHFileLink" creation_time: 1675983320650 type: FLUSH version: 2 ttl: 0 ) in 530 msec 2023-02-09 22:55:37,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 22:55:37,896 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-09 22:55:37,896 INFO [Listener at localhost/33819] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:37,900 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(748): Client=jenkins//172.31.2.10 delete name: "hbase-failsafe-snaptb-1675983320640-1675983335127" 2023-02-09 22:55:37,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1675983320640-1675983335127 2023-02-09 22:55:37,904 INFO [Listener at localhost/33819] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-09 22:55:37,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.HMaster$10(2372): Client=jenkins//172.31.2.10 enable testCompactionWithHFileLink 2023-02-09 22:55:37,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-09 22:55:37,916 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-09 22:55:37,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 22:55:37,916 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983337916"}]},"ts":"1675983337916"} 2023-02-09 22:55:37,917 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-09 22:55:37,920 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-09 22:55:37,920 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN}] 2023-02-09 22:55:37,921 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN 2023-02-09 22:55:37,921 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN 2023-02-09 22:55:37,921 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN 2023-02-09 22:55:37,922 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 22:55:37,922 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 22:55:37,922 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-09 22:55:38,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 22:55:38,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,074 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983338074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983338074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983338074"}]},"ts":"1675983338074"} 2023-02-09 22:55:38,074 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,074 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983338074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983338074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983338074"}]},"ts":"1675983338074"} 2023-02-09 22:55:38,074 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983338074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983338074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983338074"}]},"ts":"1675983338074"} 2023-02-09 22:55:38,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:38,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:38,077 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:38,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 22:55:38,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:38,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 644cc6a94a1dc8fa8754303752b40531, NAME => 'testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 22:55:38,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:38,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,233 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,234 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:38,234 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:38,235 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 644cc6a94a1dc8fa8754303752b40531 columnFamilyName family1 2023-02-09 22:55:38,252 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2 2023-02-09 22:55:38,258 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a 2023-02-09 22:55:38,264 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0 2023-02-09 22:55:38,269 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6 2023-02-09 22:55:38,269 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(310): Store=644cc6a94a1dc8fa8754303752b40531/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,270 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,271 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2 2023-02-09 22:55:38,271 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2 2023-02-09 22:55:38,271 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 644cc6a94a1dc8fa8754303752b40531 columnFamilyName family2 2023-02-09 22:55:38,278 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f 2023-02-09 22:55:38,286 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061 2023-02-09 22:55:38,292 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4 2023-02-09 22:55:38,299 DEBUG [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6 2023-02-09 22:55:38,299 INFO [StoreOpener-644cc6a94a1dc8fa8754303752b40531-1] regionserver.HStore(310): Store=644cc6a94a1dc8fa8754303752b40531/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:38,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:55:38,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 644cc6a94a1dc8fa8754303752b40531; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65177181, jitterRate=-0.028784319758415222}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:38,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 644cc6a94a1dc8fa8754303752b40531: Running coprocessor pre-open hook at 1675983338232Writing region info on filesystem at 1675983338232Initializing all the Stores at 1675983338233 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338233Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338233Cleaning up temporary data from old regions at 1675983338301 (+68 ms)Cleaning up detritus from prior splits at 1675983338302 (+1 ms)Running coprocessor post-open hooks at 1675983338304 (+2 ms)Region opened successfully at 1675983338304 2023-02-09 22:55:38,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., pid=101, masterSystemTime=1675983338229 2023-02-09 22:55:38,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:38,305 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:55:38,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:38,306 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:55:38,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:38,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:38,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:38,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1bc7ec556ab621014c7a07a6e844bff2, NAME => 'testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 22:55:38,308 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=644cc6a94a1dc8fa8754303752b40531, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,308 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983338308"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983338308"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983338308"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983338308"}]},"ts":"1675983338308"} 2023-02-09 22:55:38,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:38,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,309 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 22:55:38,309 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1912): 644cc6a94a1dc8fa8754303752b40531/family1 is initiating minor compaction (all files) 2023-02-09 22:55:38,309 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 644cc6a94a1dc8fa8754303752b40531/family1 in testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:38,309 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp, totalSize=21.6 K 2023-02-09 22:55:38,310 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:38,310 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 22:55:38,311 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1912): 644cc6a94a1dc8fa8754303752b40531/family2 is initiating minor compaction (all files) 2023-02-09 22:55:38,311 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 644cc6a94a1dc8fa8754303752b40531/family2 in testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:55:38,311 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp, totalSize=20.4 K 2023-02-09 22:55:38,311 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,311 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-02-09 22:55:38,311 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:38,311 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 644cc6a94a1dc8fa8754303752b40531, server=jenkins-hbase9.apache.org,39093,1675983215026 in 234 msec 2023-02-09 22:55:38,312 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:38,312 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:38,312 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:38,313 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:38,313 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bc7ec556ab621014c7a07a6e844bff2 columnFamilyName family1 2023-02-09 22:55:38,313 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=644cc6a94a1dc8fa8754303752b40531, ASSIGN in 392 msec 2023-02-09 22:55:38,313 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:38,313 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:38,314 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:38,315 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:38,330 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee 2023-02-09 22:55:38,343 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0 2023-02-09 22:55:38,349 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 644cc6a94a1dc8fa8754303752b40531#family1#compaction#114 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:38,349 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] throttle.PressureAwareThroughputController(145): 644cc6a94a1dc8fa8754303752b40531#family2#compaction#115 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:38,352 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f 2023-02-09 22:55:38,363 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6 2023-02-09 22:55:38,364 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(310): Store=1bc7ec556ab621014c7a07a6e844bff2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,364 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,365 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2 2023-02-09 22:55:38,365 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2 2023-02-09 22:55:38,365 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1bc7ec556ab621014c7a07a6e844bff2 columnFamilyName family2 2023-02-09 22:55:38,370 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a 2023-02-09 22:55:38,374 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5 2023-02-09 22:55:38,378 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba 2023-02-09 22:55:38,382 DEBUG [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad 2023-02-09 22:55:38,382 INFO [StoreOpener-1bc7ec556ab621014c7a07a6e844bff2-1] regionserver.HStore(310): Store=1bc7ec556ab621014c7a07a6e844bff2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:38,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:55:38,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 1bc7ec556ab621014c7a07a6e844bff2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72128175, jitterRate=0.07479356229305267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:38,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 1bc7ec556ab621014c7a07a6e844bff2: Running coprocessor pre-open hook at 1675983338309Writing region info on filesystem at 1675983338309Initializing all the Stores at 1675983338310 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338310Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338311 (+1 ms)Cleaning up temporary data from old regions at 1675983338384 (+73 ms)Cleaning up detritus from prior splits at 1675983338385 (+1 ms)Running coprocessor post-open hooks at 1675983338387 (+2 ms)Region opened successfully at 1675983338387 2023-02-09 22:55:38,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., pid=103, masterSystemTime=1675983338229 2023-02-09 22:55:38,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 22:55:38,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:38,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:38,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d030c0c1080b19b33ee54050ad4ea0a2, NAME => 'testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 22:55:38,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=1bc7ec556ab621014c7a07a6e844bff2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:38,389 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675983338389"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983338389"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983338389"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983338389"}]},"ts":"1675983338389"} 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,391 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,391 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:38,391 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:38,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-02-09 22:55:38,392 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d030c0c1080b19b33ee54050ad4ea0a2 columnFamilyName family1 2023-02-09 22:55:38,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 1bc7ec556ab621014c7a07a6e844bff2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 313 msec 2023-02-09 22:55:38,393 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1bc7ec556ab621014c7a07a6e844bff2, ASSIGN in 472 msec 2023-02-09 22:55:38,399 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd 2023-02-09 22:55:38,404 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0 2023-02-09 22:55:38,411 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2 2023-02-09 22:55:38,417 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90 2023-02-09 22:55:38,419 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(310): Store=d030c0c1080b19b33ee54050ad4ea0a2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,419 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,420 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2 2023-02-09 22:55:38,420 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2 2023-02-09 22:55:38,421 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d030c0c1080b19b33ee54050ad4ea0a2 columnFamilyName family2 2023-02-09 22:55:38,429 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98 2023-02-09 22:55:38,439 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7 2023-02-09 22:55:38,447 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d 2023-02-09 22:55:38,460 DEBUG [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(539): loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91 2023-02-09 22:55:38,461 INFO [StoreOpener-d030c0c1080b19b33ee54050ad4ea0a2-1] regionserver.HStore(310): Store=d030c0c1080b19b33ee54050ad4ea0a2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:38,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:38,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:55:38,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened d030c0c1080b19b33ee54050ad4ea0a2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62830728, jitterRate=-0.06374919414520264}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:38,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for d030c0c1080b19b33ee54050ad4ea0a2: Running coprocessor pre-open hook at 1675983338390Writing region info on filesystem at 1675983338390Initializing all the Stores at 1675983338390Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338390Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983338390Cleaning up temporary data from old regions at 1675983338465 (+75 ms)Cleaning up detritus from prior splits at 1675983338465Running coprocessor post-open hooks at 1675983338468 (+3 ms)Region opened successfully at 1675983338468 2023-02-09 22:55:38,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., pid=102, masterSystemTime=1675983338229 2023-02-09 22:55:38,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 22:55:38,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 22:55:38,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:38,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:38,471 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=d030c0c1080b19b33ee54050ad4ea0a2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:38,471 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983338471"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983338471"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983338471"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983338471"}]},"ts":"1675983338471"} 2023-02-09 22:55:38,475 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=99 2023-02-09 22:55:38,475 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure d030c0c1080b19b33ee54050ad4ea0a2, server=jenkins-hbase9.apache.org,39093,1675983215026 in 397 msec 2023-02-09 22:55:38,477 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-02-09 22:55:38,477 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d030c0c1080b19b33ee54050ad4ea0a2, ASSIGN in 555 msec 2023-02-09 22:55:38,477 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983338477"}]},"ts":"1675983338477"} 2023-02-09 22:55:38,479 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-09 22:55:38,481 INFO [PEWorker-3] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-09 22:55:38,483 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 572 msec 2023-02-09 22:55:38,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-09 22:55:38,519 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-09 22:55:38,532 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:38,541 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:38,548 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:38,549 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:38,556 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:55:38,569 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-09 22:55:38,570 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:38,570 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(139): Connect 0x5ea8e728 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:38,574 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c4eeca5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:38,574 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:38,574 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:38,574 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:38,665 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020946138f424f11433a88ff6a30c24768a6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020946138f424f11433a88ff6a30c24768a6 2023-02-09 22:55:38,666 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:38,679 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:38,683 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:34942, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:38,690 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:38,692 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58234, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:38,712 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/3224436ffeb943ca88099282becc1161 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-09 22:55:38,713 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/3224436ffeb943ca88099282becc1161}] 2023-02-09 22:55:38,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/3224436ffeb943ca88099282becc1161 for inclusion in d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:38,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-09 22:55:38,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 22:55:38,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:38,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209/family1/3224436ffeb943ca88099282becc1161 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__q7ld85t4o38j7l4v29g0kg1e148ljnm3ccpesh5c9ql3t9o1h1m1bf6ifctqp6qc/family1/3224436ffeb943ca88099282becc1161 2023-02-09 22:55:38,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__q7ld85t4o38j7l4v29g0kg1e148ljnm3ccpesh5c9ql3t9o1h1m1bf6ifctqp6qc/family1/3224436ffeb943ca88099282becc1161 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_ 2023-02-09 22:55:38,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__q7ld85t4o38j7l4v29g0kg1e148ljnm3ccpesh5c9ql3t9o1h1m1bf6ifctqp6qc/family1/3224436ffeb943ca88099282becc1161 into d030c0c1080b19b33ee54050ad4ea0a2/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_ - updating store file list. 2023-02-09 22:55:38,735 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_ into d030c0c1080b19b33ee54050ad4ea0a2/family1 2023-02-09 22:55:38,735 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__q7ld85t4o38j7l4v29g0kg1e148ljnm3ccpesh5c9ql3t9o1h1m1bf6ifctqp6qc/family1/3224436ffeb943ca88099282becc1161 into d030c0c1080b19b33ee54050ad4ea0a2/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_) 2023-02-09 22:55:38,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__q7ld85t4o38j7l4v29g0kg1e148ljnm3ccpesh5c9ql3t9o1h1m1bf6ifctqp6qc/family1/3224436ffeb943ca88099282becc1161 2023-02-09 22:55:38,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:38,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-09 22:55:38,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(1912): d030c0c1080b19b33ee54050ad4ea0a2/family1 is initiating minor compaction (all files) 2023-02-09 22:55:38,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5bcb6e42; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-09 22:55:38,739 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:38,740 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:38,741 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020962a983017fbe4553a7b3bd87b35374eb 2023-02-09 22:55:38,742 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302097e941376e984481ebf9628e59fdc6ef8 2023-02-09 22:55:38,743 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020997d1fd66cd644fec99d99d51b41ac9c4 2023-02-09 22:55:38,744 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230209c9dfb47d3d8d497397931530db9a2a8f 2023-02-09 22:55:38,744 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:38,744 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 22:55:38,744 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:39,469 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209886a472460ff4d839ff6a8f3651af1cc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230209886a472460ff4d839ff6a8f3651af1cc 2023-02-09 22:55:39,469 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209ed8a43e547094dc28f4565e629fee6f1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230209ed8a43e547094dc28f4565e629fee6f1 2023-02-09 22:55:39,469 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:39,469 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:39,494 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/e01611029a18479499f50a0df820af1a first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-09 22:55:39,494 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/e01611029a18479499f50a0df820af1a}] 2023-02-09 22:55:39,494 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/12a543c92520419f9036008d2209b37f first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-09 22:55:39,495 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=32 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/12a543c92520419f9036008d2209b37f}] 2023-02-09 22:55:39,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/e01611029a18479499f50a0df820af1a for inclusion in 644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:39,497 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/12a543c92520419f9036008d2209b37f for inclusion in 1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:39,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:39,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209/family1/12a543c92520419f9036008d2209b37f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__v162velkbpa835dgimr0si60d8sboc2esbkdmmaehol0g8hf15c832k0b1voinkf/family1/12a543c92520419f9036008d2209b37f 2023-02-09 22:55:39,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209/family1/e01611029a18479499f50a0df820af1a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__29k98i8mqah6m8pq9d8ku66muc3j4rldgkoug98i5ci3focd2c1j1tehuuke0pr2/family1/e01611029a18479499f50a0df820af1a 2023-02-09 22:55:39,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__v162velkbpa835dgimr0si60d8sboc2esbkdmmaehol0g8hf15c832k0b1voinkf/family1/12a543c92520419f9036008d2209b37f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_ 2023-02-09 22:55:39,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__29k98i8mqah6m8pq9d8ku66muc3j4rldgkoug98i5ci3focd2c1j1tehuuke0pr2/family1/e01611029a18479499f50a0df820af1a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/da5570283fa248b98ffb838c937a0d23_SeqId_34_ 2023-02-09 22:55:39,503 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__v162velkbpa835dgimr0si60d8sboc2esbkdmmaehol0g8hf15c832k0b1voinkf/family1/12a543c92520419f9036008d2209b37f into 1bc7ec556ab621014c7a07a6e844bff2/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_ - updating store file list. 2023-02-09 22:55:39,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__29k98i8mqah6m8pq9d8ku66muc3j4rldgkoug98i5ci3focd2c1j1tehuuke0pr2/family1/e01611029a18479499f50a0df820af1a into 644cc6a94a1dc8fa8754303752b40531/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/da5570283fa248b98ffb838c937a0d23_SeqId_34_ - updating store file list. 2023-02-09 22:55:39,506 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_ into 1bc7ec556ab621014c7a07a6e844bff2/family1 2023-02-09 22:55:39,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__v162velkbpa835dgimr0si60d8sboc2esbkdmmaehol0g8hf15c832k0b1voinkf/family1/12a543c92520419f9036008d2209b37f into 1bc7ec556ab621014c7a07a6e844bff2/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_) 2023-02-09 22:55:39,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/da5570283fa248b98ffb838c937a0d23_SeqId_34_ into 644cc6a94a1dc8fa8754303752b40531/family1 2023-02-09 22:55:39,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__29k98i8mqah6m8pq9d8ku66muc3j4rldgkoug98i5ci3focd2c1j1tehuuke0pr2/family1/e01611029a18479499f50a0df820af1a into 644cc6a94a1dc8fa8754303752b40531/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/da5570283fa248b98ffb838c937a0d23_SeqId_34_) 2023-02-09 22:55:39,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__v162velkbpa835dgimr0si60d8sboc2esbkdmmaehol0g8hf15c832k0b1voinkf/family1/12a543c92520419f9036008d2209b37f 2023-02-09 22:55:39,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testCompactionWithHFileLink__29k98i8mqah6m8pq9d8ku66muc3j4rldgkoug98i5ci3focd2c1j1tehuuke0pr2/family1/e01611029a18479499f50a0df820af1a 2023-02-09 22:55:39,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:39,509 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:39,509 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:39,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-09 22:55:39,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(1912): 1bc7ec556ab621014c7a07a6e844bff2/family1 is initiating minor compaction (all files) 2023-02-09 22:55:39,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@50bccf48; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-09 22:55:39,511 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302097b8f07926b614ad2a4ffcaf307146348 2023-02-09 22:55:39,512 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:39,513 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209a4ce4c9ca54043e9b796aafda9414f1b 2023-02-09 22:55:39,513 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:39,514 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209dcd935d183784441a3eb42d91bcf8acc 2023-02-09 22:55:39,514 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302091288e3c75e3c49b8ba33ef35065a9797 2023-02-09 22:55:39,515 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230209ec0c2b8f283643129caa6c5d25287a80 2023-02-09 22:55:39,515 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:39,515 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 22:55:39,516 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209afb269af56cd42f9819767d554fca42f 2023-02-09 22:55:39,516 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:39,517 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209c5806035e5974e3fa508cdec5d65b40f 2023-02-09 22:55:39,518 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230209dfaf4a1e04f8431e9b59a5973a5d5dd5 2023-02-09 22:55:39,518 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-09 22:55:39,518 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 22:55:39,518 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testCompactionWithHFileLink' and column='family1' 2023-02-09 22:55:39,518 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-09 22:55:39,521 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:39,523 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:39,526 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:39,527 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:39,533 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:39,534 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:39,551 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=283 (was 244) - Thread LEAK? -, OpenFileDescriptor=755 (was 615) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 104), ProcessCount=168 (was 168), AvailableMemoryMB=10665 (was 10746) 2023-02-09 22:55:39,566 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=283, OpenFileDescriptor=755, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=168, AvailableMemoryMB=10664 2023-02-09 22:55:39,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$15(2997): Client=jenkins//172.31.2.10 creating {NAME => 'ns'} 2023-02-09 22:55:39,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-09 22:55:39,574 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family1/e4e7dcfa4f2e42439fa15e8620f3594d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/e4e7dcfa4f2e42439fa15e8620f3594d 2023-02-09 22:55:39,576 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/.tmp/family2/1f300ecebf2c4ca2a9f97427aa9504d6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/1f300ecebf2c4ca2a9f97427aa9504d6 2023-02-09 22:55:39,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 22:55:39,580 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-09 22:55:39,580 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 22:55:39,580 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 644cc6a94a1dc8fa8754303752b40531/family1 of 644cc6a94a1dc8fa8754303752b40531 into e4e7dcfa4f2e42439fa15e8620f3594d(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:39,580 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:39,580 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., storeName=644cc6a94a1dc8fa8754303752b40531/family1, priority=12, startTime=1675983338305; duration=1sec 2023-02-09 22:55:39,580 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-09 22:55:39,581 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d030c0c1080b19b33ee54050ad4ea0a2/family1 in testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:39,581 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp, totalSize=28.9 K 2023-02-09 22:55:39,581 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 644cc6a94a1dc8fa8754303752b40531/family2 of 644cc6a94a1dc8fa8754303752b40531 into 1f300ecebf2c4ca2a9f97427aa9504d6(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:39,581 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 644cc6a94a1dc8fa8754303752b40531: 2023-02-09 22:55:39,581 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531., storeName=644cc6a94a1dc8fa8754303752b40531/family2, priority=12, startTime=1675983338305; duration=1sec 2023-02-09 22:55:39,582 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-09 22:55:39,582 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1bc7ec556ab621014c7a07a6e844bff2/family1 in testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:39,582 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp, totalSize=28.9 K 2023-02-09 22:55:39,582 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:39,583 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313487 2023-02-09 22:55:39,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-02-09 22:55:39,583 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:39,583 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:39,583 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:39,584 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:39,584 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:39,584 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675983313489 2023-02-09 22:55:39,584 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:39,585 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting f7704bde701945aa801bb19ac09ac2f7_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675983313487 2023-02-09 22:55:39,597 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): d030c0c1080b19b33ee54050ad4ea0a2#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:39,597 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] throttle.PressureAwareThroughputController(145): 1bc7ec556ab621014c7a07a6e844bff2#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:39,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2] to archive 2023-02-09 22:55:39,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:39,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-4e7a9faf25f04cfea21bb52c9ed5615a 2023-02-09 22:55:39,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-6caab206f3eb4f499669f0c3f437a6b0 2023-02-09 22:55:39,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-fcc50ba542b04c1cab2f19bc13ca18d6 2023-02-09 22:55:39,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family1/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-19c40713542541c0b2676be0471061d2 2023-02-09 22:55:39,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6] to archive 2023-02-09 22:55:39,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:39,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-109596cc15d14cfea976ed9bedf2377f 2023-02-09 22:55:39,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-94ad938e948e4b0eb64085e2a5d251d4 2023-02-09 22:55:39,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-26a11428bcba441ead1aecdbafb98061 2023-02-09 22:55:39,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/family2/testCompactionWithHFileLink=644cc6a94a1dc8fa8754303752b40531-d84bf35b04104c36b3a0c293372631c6 2023-02-09 22:55:39,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-09 22:55:39,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 22:55:39,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-09 22:55:39,684 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:39,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-09 22:55:39,684 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:39,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:39,686 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5 empty. 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7 empty. 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a empty. 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:39,687 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:39,688 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-09 22:55:39,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:39,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:40,095 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:40,096 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 4e0b246c234b9e9ec553bf509305303a, NAME => 'ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:40,096 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3348a4e2ede73e060506e7ba945c70b5, NAME => 'ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:40,096 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => 6efa5ac2e6e42332f71fcc029b1955d7, NAME => 'ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing 6efa5ac2e6e42332f71fcc029b1955d7, disabling compactions & flushes 2023-02-09 22:55:40,105 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. after waiting 0 ms 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,105 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,105 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for 6efa5ac2e6e42332f71fcc029b1955d7: Waiting for close lock at 1675983340105Disabling compacts and flushes for region at 1675983340105Disabling writes for close at 1675983340105Writing region close event to WAL at 1675983340105Closed at 1675983340105 2023-02-09 22:55:40,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:40,361 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:40,416 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family1/9ce220af3ab04022b632f58b2d2c41db as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/9ce220af3ab04022b632f58b2d2c41db 2023-02-09 22:55:40,420 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 1bc7ec556ab621014c7a07a6e844bff2/family1 of 1bc7ec556ab621014c7a07a6e844bff2 into 9ce220af3ab04022b632f58b2d2c41db(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:40,420 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:40,420 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675983339510; duration=0sec 2023-02-09 22:55:40,421 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-09 22:55:40,421 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:55:40,423 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 22:55:40,423 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1912): 1bc7ec556ab621014c7a07a6e844bff2/family2 is initiating minor compaction (all files) 2023-02-09 22:55:40,423 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 1bc7ec556ab621014c7a07a6e844bff2/family2 in testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:55:40,423 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp, totalSize=20.4 K 2023-02-09 22:55:40,424 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313487 2023-02-09 22:55:40,425 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:40,425 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:40,426 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:40,438 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] throttle.PressureAwareThroughputController(145): 1bc7ec556ab621014c7a07a6e844bff2#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 4e0b246c234b9e9ec553bf509305303a, disabling compactions & flushes 2023-02-09 22:55:40,505 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. after waiting 0 ms 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,505 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,505 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 4e0b246c234b9e9ec553bf509305303a: Waiting for close lock at 1675983340505Disabling compacts and flushes for region at 1675983340505Disabling writes for close at 1675983340505Writing region close event to WAL at 1675983340505Closed at 1675983340505 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 3348a4e2ede73e060506e7ba945c70b5, disabling compactions & flushes 2023-02-09 22:55:40,506 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. after waiting 0 ms 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,506 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,506 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 3348a4e2ede73e060506e7ba945c70b5: Waiting for close lock at 1675983340506Disabling compacts and flushes for region at 1675983340506Disabling writes for close at 1675983340506Writing region close event to WAL at 1675983340506Closed at 1675983340506 2023-02-09 22:55:40,509 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:40,510 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675983340510"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983340510"}]},"ts":"1675983340510"} 2023-02-09 22:55:40,510 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340510"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983340510"}]},"ts":"1675983340510"} 2023-02-09 22:55:40,510 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340510"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983340510"}]},"ts":"1675983340510"} 2023-02-09 22:55:40,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_] to archive 2023-02-09 22:55:40,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:40,515 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 22:55:40,516 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:40,516 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983340516"}]},"ts":"1675983340516"} 2023-02-09 22:55:40,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-c85ef1364b9e4b06abc3c6c00f1d96ee 2023-02-09 22:55:40,517 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-09 22:55:40,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-cd51f7493e8d4aa19d1ec014571098c0 2023-02-09 22:55:40,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-efe9f66995494ba78d74ee5225599fb6 2023-02-09 22:55:40,522 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3348a4e2ede73e060506e7ba945c70b5, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6efa5ac2e6e42332f71fcc029b1955d7, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4e0b246c234b9e9ec553bf509305303a, ASSIGN}] 2023-02-09 22:55:40,524 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3348a4e2ede73e060506e7ba945c70b5, ASSIGN 2023-02-09 22:55:40,524 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6efa5ac2e6e42332f71fcc029b1955d7, ASSIGN 2023-02-09 22:55:40,524 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4e0b246c234b9e9ec553bf509305303a, ASSIGN 2023-02-09 22:55:40,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-dae3e28db95e402782a4642a708b9b8f 2023-02-09 22:55:40,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family1/f7704bde701945aa801bb19ac09ac2f7_SeqId_34_ 2023-02-09 22:55:40,527 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3348a4e2ede73e060506e7ba945c70b5, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:40,527 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6efa5ac2e6e42332f71fcc029b1955d7, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:40,528 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4e0b246c234b9e9ec553bf509305303a, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:40,680 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3348a4e2ede73e060506e7ba945c70b5, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,680 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=6efa5ac2e6e42332f71fcc029b1955d7, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=4e0b246c234b9e9ec553bf509305303a, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,680 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340680"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983340680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983340680"}]},"ts":"1675983340680"} 2023-02-09 22:55:40,680 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675983340680"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983340680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983340680"}]},"ts":"1675983340680"} 2023-02-09 22:55:40,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340680"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983340680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983340680"}]},"ts":"1675983340680"} 2023-02-09 22:55:40,681 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure 3348a4e2ede73e060506e7ba945c70b5, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:40,682 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 6efa5ac2e6e42332f71fcc029b1955d7, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:40,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 4e0b246c234b9e9ec553bf509305303a, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:40,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:40,817 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family1/a24621bd843b48f4b3430d4125d49d44 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/a24621bd843b48f4b3430d4125d49d44 2023-02-09 22:55:40,821 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in d030c0c1080b19b33ee54050ad4ea0a2/family1 of d030c0c1080b19b33ee54050ad4ea0a2 into a24621bd843b48f4b3430d4125d49d44(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:40,821 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:40,821 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675983338737; duration=1sec 2023-02-09 22:55:40,822 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-09 22:55:40,822 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 22:55:40,824 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-09 22:55:40,824 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1912): d030c0c1080b19b33ee54050ad4ea0a2/family2 is initiating minor compaction (all files) 2023-02-09 22:55:40,824 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d030c0c1080b19b33ee54050ad4ea0a2/family2 in testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:55:40,824 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp, totalSize=20.4 K 2023-02-09 22:55:40,824 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983313489 2023-02-09 22:55:40,825 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983315274 2023-02-09 22:55:40,826 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983317069 2023-02-09 22:55:40,827 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983318855 2023-02-09 22:55:40,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3348a4e2ede73e060506e7ba945c70b5, NAME => 'ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 22:55:40,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,838 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,839 DEBUG [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1 2023-02-09 22:55:40,840 DEBUG [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1 2023-02-09 22:55:40,840 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3348a4e2ede73e060506e7ba945c70b5 columnFamilyName family1 2023-02-09 22:55:40,842 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] regionserver.HStore(310): Store=3348a4e2ede73e060506e7ba945c70b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,842 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,844 DEBUG [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2 2023-02-09 22:55:40,844 DEBUG [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2 2023-02-09 22:55:40,844 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3348a4e2ede73e060506e7ba945c70b5 columnFamilyName family2 2023-02-09 22:55:40,850 INFO [StoreOpener-3348a4e2ede73e060506e7ba945c70b5-1] regionserver.HStore(310): Store=3348a4e2ede73e060506e7ba945c70b5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:40,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:55:40,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:40,867 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): d030c0c1080b19b33ee54050ad4ea0a2#family2#compaction#119 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:40,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 3348a4e2ede73e060506e7ba945c70b5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74094812, jitterRate=0.10409873723983765}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:40,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 3348a4e2ede73e060506e7ba945c70b5: Running coprocessor pre-open hook at 1675983340835Writing region info on filesystem at 1675983340835Initializing all the Stores at 1675983340836 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340837 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340838 (+1 ms)Cleaning up temporary data from old regions at 1675983340853 (+15 ms)Cleaning up detritus from prior splits at 1675983340853Running coprocessor post-open hooks at 1675983340867 (+14 ms)Region opened successfully at 1675983340867 2023-02-09 22:55:40,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5., pid=109, masterSystemTime=1675983340832 2023-02-09 22:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:40,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6efa5ac2e6e42332f71fcc029b1955d7, NAME => 'ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 22:55:40,869 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3348a4e2ede73e060506e7ba945c70b5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,870 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340869"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983340869"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983340869"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983340869"}]},"ts":"1675983340869"} 2023-02-09 22:55:40,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,872 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-02-09 22:55:40,872 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure 3348a4e2ede73e060506e7ba945c70b5, server=jenkins-hbase9.apache.org,39093,1675983215026 in 190 msec 2023-02-09 22:55:40,873 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3348a4e2ede73e060506e7ba945c70b5, ASSIGN in 350 msec 2023-02-09 22:55:40,875 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,883 DEBUG [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1 2023-02-09 22:55:40,883 DEBUG [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1 2023-02-09 22:55:40,883 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6efa5ac2e6e42332f71fcc029b1955d7 columnFamilyName family1 2023-02-09 22:55:40,884 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] regionserver.HStore(310): Store=6efa5ac2e6e42332f71fcc029b1955d7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,884 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,885 DEBUG [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2 2023-02-09 22:55:40,885 DEBUG [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2 2023-02-09 22:55:40,886 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6efa5ac2e6e42332f71fcc029b1955d7 columnFamilyName family2 2023-02-09 22:55:40,886 INFO [StoreOpener-6efa5ac2e6e42332f71fcc029b1955d7-1] regionserver.HStore(310): Store=6efa5ac2e6e42332f71fcc029b1955d7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:55:40,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:40,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 6efa5ac2e6e42332f71fcc029b1955d7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68399090, jitterRate=0.019225865602493286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:40,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 6efa5ac2e6e42332f71fcc029b1955d7: Running coprocessor pre-open hook at 1675983340870Writing region info on filesystem at 1675983340870Initializing all the Stores at 1675983340870Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340870Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340875 (+5 ms)Cleaning up temporary data from old regions at 1675983340888 (+13 ms)Cleaning up detritus from prior splits at 1675983340888Running coprocessor post-open hooks at 1675983340891 (+3 ms)Region opened successfully at 1675983340891 2023-02-09 22:55:40,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7., pid=110, masterSystemTime=1675983340832 2023-02-09 22:55:40,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:40,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4e0b246c234b9e9ec553bf509305303a, NAME => 'ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 22:55:40,893 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=6efa5ac2e6e42332f71fcc029b1955d7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,893 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675983340893"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983340893"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983340893"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983340893"}]},"ts":"1675983340893"} 2023-02-09 22:55:40,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:40,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,895 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,896 DEBUG [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1 2023-02-09 22:55:40,896 DEBUG [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1 2023-02-09 22:55:40,896 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-02-09 22:55:40,896 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 6efa5ac2e6e42332f71fcc029b1955d7, server=jenkins-hbase9.apache.org,39093,1675983215026 in 213 msec 2023-02-09 22:55:40,896 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e0b246c234b9e9ec553bf509305303a columnFamilyName family1 2023-02-09 22:55:40,897 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6efa5ac2e6e42332f71fcc029b1955d7, ASSIGN in 374 msec 2023-02-09 22:55:40,897 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] regionserver.HStore(310): Store=4e0b246c234b9e9ec553bf509305303a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,897 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,898 DEBUG [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2 2023-02-09 22:55:40,898 DEBUG [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2 2023-02-09 22:55:40,898 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e0b246c234b9e9ec553bf509305303a columnFamilyName family2 2023-02-09 22:55:40,899 INFO [StoreOpener-4e0b246c234b9e9ec553bf509305303a-1] regionserver.HStore(310): Store=4e0b246c234b9e9ec553bf509305303a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:40,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:55:40,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:55:40,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:40,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 4e0b246c234b9e9ec553bf509305303a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65289032, jitterRate=-0.027117609977722168}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:55:40,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 4e0b246c234b9e9ec553bf509305303a: Running coprocessor pre-open hook at 1675983340894Writing region info on filesystem at 1675983340894Initializing all the Stores at 1675983340894Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340894Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983340894Cleaning up temporary data from old regions at 1675983340900 (+6 ms)Cleaning up detritus from prior splits at 1675983340901 (+1 ms)Running coprocessor post-open hooks at 1675983340903 (+2 ms)Region opened successfully at 1675983340903 2023-02-09 22:55:40,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a., pid=111, masterSystemTime=1675983340832 2023-02-09 22:55:40,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:40,905 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=4e0b246c234b9e9ec553bf509305303a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:40,905 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675983340905"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983340905"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983340905"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983340905"}]},"ts":"1675983340905"} 2023-02-09 22:55:40,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-02-09 22:55:40,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 4e0b246c234b9e9ec553bf509305303a, server=jenkins-hbase9.apache.org,39093,1675983215026 in 224 msec 2023-02-09 22:55:40,909 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-02-09 22:55:40,909 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4e0b246c234b9e9ec553bf509305303a, ASSIGN in 385 msec 2023-02-09 22:55:40,909 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:40,909 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983340909"}]},"ts":"1675983340909"} 2023-02-09 22:55:40,910 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-09 22:55:40,912 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:40,913 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2310 sec 2023-02-09 22:55:40,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_] to archive 2023-02-09 22:55:40,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:40,924 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-55831fc6ce994a5cb81893af3b5f34fd 2023-02-09 22:55:40,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-978c8825afba40b3a59aa1565689d9c0 2023-02-09 22:55:40,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-cac9ab38699a4e2fac99af68222a3f90 2023-02-09 22:55:40,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-b4225150012c4e1f92b711ac1d5ccae2 2023-02-09 22:55:40,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family1/dcc1e389a9414ddbb4171d5a2cfd0309_SeqId_34_ 2023-02-09 22:55:41,305 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/.tmp/family2/8590f5a572b64cf99a58788b87707b24 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/8590f5a572b64cf99a58788b87707b24 2023-02-09 22:55:41,311 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d030c0c1080b19b33ee54050ad4ea0a2/family2 of d030c0c1080b19b33ee54050ad4ea0a2 into 8590f5a572b64cf99a58788b87707b24(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:41,311 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d030c0c1080b19b33ee54050ad4ea0a2: 2023-02-09 22:55:41,311 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2., storeName=d030c0c1080b19b33ee54050ad4ea0a2/family2, priority=12, startTime=1675983338469; duration=0sec 2023-02-09 22:55:41,311 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 22:55:41,311 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. because compaction request was cancelled 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 22:55:41,312 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. because compaction request was cancelled 2023-02-09 22:55:41,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d] to archive 2023-02-09 22:55:41,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:41,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-e90eb10934884f2592db3b80bd585a91 2023-02-09 22:55:41,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-74769bcc3f08490d9e45153b83bc91a7 2023-02-09 22:55:41,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-13ff32351eca42509c4dbfadc1f72c98 2023-02-09 22:55:41,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/family2/testCompactionWithHFileLink=d030c0c1080b19b33ee54050ad4ea0a2-c49570f4da0647cfae6e795d461a515d 2023-02-09 22:55:41,660 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/.tmp/family2/cfa2d9d669b342dfb1be40062078847a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/cfa2d9d669b342dfb1be40062078847a 2023-02-09 22:55:41,663 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1bc7ec556ab621014c7a07a6e844bff2/family2 of 1bc7ec556ab621014c7a07a6e844bff2 into cfa2d9d669b342dfb1be40062078847a(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:41,663 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1bc7ec556ab621014c7a07a6e844bff2: 2023-02-09 22:55:41,663 INFO [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2., storeName=1bc7ec556ab621014c7a07a6e844bff2/family2, priority=12, startTime=1675983338388; duration=1sec 2023-02-09 22:55:41,664 DEBUG [RS:0;jenkins-hbase9:39093-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:55:41,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad] to archive 2023-02-09 22:55:41,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:41,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-711ccfffc8ca4f1f8a6d9a02c4efbcba 2023-02-09 22:55:41,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-5dd6f138a437444facbaa6253289b6f5 2023-02-09 22:55:41,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-45b2586e54a4454b8ea853e99eb3525a 2023-02-09 22:55:41,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/family2/testCompactionWithHFileLink=1bc7ec556ab621014c7a07a6e844bff2-ffdb38aa331c4ba9ab6cbceef95237ad 2023-02-09 22:55:41,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 22:55:41,789 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-09 22:55:41,795 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:41,796 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:41,796 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:41,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:41,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 22:55:41,799 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 22:55:41,799 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:41,799 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 22:55:41,799 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 22:55:41,800 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:41,800 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:41,801 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:41,801 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:41,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:41,801 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:41,801 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:41,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:41,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 22:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 22:55:41,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 22:55:41,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:41,802 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 22:55:41,803 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:41,803 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:41,803 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:41,803 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. started... 2023-02-09 22:55:41,803 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:41,803 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:41,803 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. started... 2023-02-09 22:55:41,804 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6efa5ac2e6e42332f71fcc029b1955d7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:41,804 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. started... 2023-02-09 22:55:41,804 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4e0b246c234b9e9ec553bf509305303a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:41,805 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3348a4e2ede73e060506e7ba945c70b5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:55:41,816 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209442bc4feec594618ba869b59a7cc6e29 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209442bc4feec594618ba869b59a7cc6e29 2023-02-09 22:55:42,216 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302092889d5e3e393497abf687380e38a5baa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302092889d5e3e393497abf687380e38a5baa 2023-02-09 22:55:42,216 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093c60cf23522d40c985bda6b8ba71b35b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302093c60cf23522d40c985bda6b8ba71b35b 2023-02-09 22:55:42,220 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/68634e7e9e6442fa81fd495443ef9c22 2023-02-09 22:55:42,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/bdb8f24621424d3c9b7c46c558390009 2023-02-09 22:55:42,621 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/c4db2118e0394b25b53ecd3b172d619b 2023-02-09 22:55:42,632 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020908e65359b7af41efafef14dae71247bf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020908e65359b7af41efafef14dae71247bf 2023-02-09 22:55:42,636 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/46aac46fd71742579ce21591d0464e6b 2023-02-09 22:55:42,640 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/68634e7e9e6442fa81fd495443ef9c22 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22 2023-02-09 22:55:42,643 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:42,644 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/46aac46fd71742579ce21591d0464e6b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b 2023-02-09 22:55:42,647 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:42,648 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6efa5ac2e6e42332f71fcc029b1955d7 in 844ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:42,648 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-09 22:55:42,648 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:42,649 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:42,649 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:43,036 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209fe3e2eec2d3e4932a3923ac04bdef313 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209fe3e2eec2d3e4932a3923ac04bdef313 2023-02-09 22:55:43,037 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302093f156f2af0724878ba0ba7e756a8f73d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302093f156f2af0724878ba0ba7e756a8f73d 2023-02-09 22:55:43,443 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/2b55321a268a4f97b12ec7032710b15d 2023-02-09 22:55:43,443 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/8a87e25671a846518bc611700695b749 2023-02-09 22:55:43,447 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/bdb8f24621424d3c9b7c46c558390009 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009 2023-02-09 22:55:43,447 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/c4db2118e0394b25b53ecd3b172d619b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b 2023-02-09 22:55:43,451 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:43,452 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b, entries=4, sequenceid=5, filesize=5.4 K 2023-02-09 22:55:43,452 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/8a87e25671a846518bc611700695b749 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749 2023-02-09 22:55:43,453 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/2b55321a268a4f97b12ec7032710b15d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d 2023-02-09 22:55:43,456 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:43,456 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:55:43,457 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4e0b246c234b9e9ec553bf509305303a in 1653ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:43,457 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3348a4e2ede73e060506e7ba945c70b5 in 1652ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:43,458 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:43,458 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:43,458 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:43,458 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 22:55:43,458 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 22:55:43,461 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,461 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:43,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:43,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,462 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:43,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:43,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:43,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:43,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 22:55:43,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77d6a309[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:43,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 22:55:43,464 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,465 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 22:55:43,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 22:55:43,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,465 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:43,465 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:43,467 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,467 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 22:55:43,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:43,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:43,467 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:43,467 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 22:55:43,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:43,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:43,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:43,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:43,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 22:55:43,470 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 22:55:43,470 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:43,470 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:43,470 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 22:55:43,470 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:43,472 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,472 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,472 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:43,472 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:43,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,473 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:43,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:43,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,477 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:43,477 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 22:55:43,477 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:43,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:43,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,478 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:43,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 22:55:43,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:43,478 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:43,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 22:55:43,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:43,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:43,579 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:43,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:43,580 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:43,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:43,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:43,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:43,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:43,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 22:55:43,585 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 22:55:43,585 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:43,585 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 22:55:43,585 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 22:55:43,585 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,585 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:43,586 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,586 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:43,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:43,586 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:43,586 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 22:55:43,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 22:55:43,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 22:55:43,587 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 22:55:43,587 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:43,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. started... 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. started... 2023-02-09 22:55:43,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:43,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:43,588 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4e0b246c234b9e9ec553bf509305303a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:43,588 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. started... 2023-02-09 22:55:43,588 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6efa5ac2e6e42332f71fcc029b1955d7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:43,588 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3348a4e2ede73e060506e7ba945c70b5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:55:43,603 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020908ca4a34d191483eaa0c90f44d220aa3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020908ca4a34d191483eaa0c90f44d220aa3 2023-02-09 22:55:44,003 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209d8a105b1067349c1951a670a768007bf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209d8a105b1067349c1951a670a768007bf 2023-02-09 22:55:44,003 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302096337f33932d24c689ea8617377886329 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302096337f33932d24c689ea8617377886329 2023-02-09 22:55:44,011 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/c03092e651ea46b8bd3ecef141931333 2023-02-09 22:55:44,014 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/4599793e540e4a52b86553dafb33fba4 2023-02-09 22:55:44,414 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/d2c2ae2e855f44a690e36276d9ad7806 2023-02-09 22:55:44,431 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020952c2498ad9874fed81ac255edaf181cf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020952c2498ad9874fed81ac255edaf181cf 2023-02-09 22:55:44,439 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209f3361f4a899841bd9e7fbb30640f0b85 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209f3361f4a899841bd9e7fbb30640f0b85 2023-02-09 22:55:44,827 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209dc044718e8de4fc6b12d538d03a25d29 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209dc044718e8de4fc6b12d538d03a25d29 2023-02-09 22:55:44,836 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/676152435ae14718a010cd5474049306 2023-02-09 22:55:44,839 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/c03092e651ea46b8bd3ecef141931333 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333 2023-02-09 22:55:44,842 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:44,843 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/676152435ae14718a010cd5474049306 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306 2023-02-09 22:55:44,844 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/67c23179259a4f74a3cd5f42fcf600cb 2023-02-09 22:55:44,846 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:44,847 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3348a4e2ede73e060506e7ba945c70b5 in 1259ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:44,847 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:44,847 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:44,847 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:44,848 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/4599793e540e4a52b86553dafb33fba4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4 2023-02-09 22:55:44,851 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:44,851 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/67c23179259a4f74a3cd5f42fcf600cb as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb 2023-02-09 22:55:44,854 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:44,855 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4e0b246c234b9e9ec553bf509305303a in 1267ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:44,855 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:44,855 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:44,855 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:45,232 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/f922065d49494886bbf5dc1a49db2cb3 2023-02-09 22:55:45,236 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/d2c2ae2e855f44a690e36276d9ad7806 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806 2023-02-09 22:55:45,239 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806, entries=4, sequenceid=9, filesize=5.4 K 2023-02-09 22:55:45,240 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/f922065d49494886bbf5dc1a49db2cb3 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3 2023-02-09 22:55:45,243 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:55:45,244 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6efa5ac2e6e42332f71fcc029b1955d7 in 1656ms, sequenceid=9, compaction requested=false 2023-02-09 22:55:45,244 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:45,244 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:45,244 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:45,244 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:45,244 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,244 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 22:55:45,244 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 22:55:45,246 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,246 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,246 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,246 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 22:55:45,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c2b1b5d[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:45,247 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 22:55:45,248 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,248 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,249 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,249 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:45,249 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 22:55:45,249 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 22:55:45,249 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:45,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 22:55:45,254 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 22:55:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:45,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 22:55:45,257 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 22:55:45,257 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:45,257 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:45,257 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 22:55:45,257 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:45,258 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,258 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,258 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,259 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:45,260 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:45,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,262 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 22:55:45,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:45,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 22:55:45,262 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:45,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,263 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:45,263 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:45,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:45,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:45,363 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:45,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:45,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:45,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:45,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:45,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:45,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:45,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 22:55:45,369 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 22:55:45,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 22:55:45,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 22:55:45,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,369 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:45,370 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,370 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:45,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:45,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:45,371 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:45,371 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 22:55:45,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 22:55:45,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 22:55:45,372 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 22:55:45,372 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:45,372 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. started... 2023-02-09 22:55:45,372 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:45,372 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. started... 2023-02-09 22:55:45,372 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6efa5ac2e6e42332f71fcc029b1955d7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:45,372 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. started... 2023-02-09 22:55:45,372 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4e0b246c234b9e9ec553bf509305303a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:45,372 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3348a4e2ede73e060506e7ba945c70b5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:55:45,785 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302096d17114887ee480eb9f6d5e6409300b6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302096d17114887ee480eb9f6d5e6409300b6 2023-02-09 22:55:45,785 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302092aea4eb297ba4771859464d169db594f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302092aea4eb297ba4771859464d169db594f 2023-02-09 22:55:45,787 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209036c203bc1824c49a6cb06a80a2be33b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209036c203bc1824c49a6cb06a80a2be33b 2023-02-09 22:55:46,191 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/302268d7ccb04e8995b09e5d0ddcd243 2023-02-09 22:55:46,191 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/36ee1a2efa6142ef9293d8861f8afbf0 2023-02-09 22:55:46,192 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/2cdc8207f3c147649eff217f7ad9d3a4 2023-02-09 22:55:46,336 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:55:46,605 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302092d99f657ec54400ea294d5251d43fbfa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302092d99f657ec54400ea294d5251d43fbfa 2023-02-09 22:55:46,605 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020965bccedfbfef495f811df49c96795a63 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023020965bccedfbfef495f811df49c96795a63 2023-02-09 22:55:46,606 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209970db56cd96f4ee9a32fc982d05ca292 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230209970db56cd96f4ee9a32fc982d05ca292 2023-02-09 22:55:47,012 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/bfeb26737f424a13a33b33a9e71d0204 2023-02-09 22:55:47,012 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/6e92e7d7dd1e4dae98c2eeb8645f0616 2023-02-09 22:55:47,012 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/8461986876c64e2183e6a6974ea43555 2023-02-09 22:55:47,016 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/36ee1a2efa6142ef9293d8861f8afbf0 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0 2023-02-09 22:55:47,016 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/302268d7ccb04e8995b09e5d0ddcd243 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243 2023-02-09 22:55:47,016 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/2cdc8207f3c147649eff217f7ad9d3a4 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4 2023-02-09 22:55:47,019 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:47,020 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:47,020 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243, entries=4, sequenceid=13, filesize=5.4 K 2023-02-09 22:55:47,020 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/6e92e7d7dd1e4dae98c2eeb8645f0616 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616 2023-02-09 22:55:47,020 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/8461986876c64e2183e6a6974ea43555 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555 2023-02-09 22:55:47,021 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/bfeb26737f424a13a33b33a9e71d0204 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204 2023-02-09 22:55:47,024 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:47,025 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:47,025 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:55:47,025 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4e0b246c234b9e9ec553bf509305303a in 1653ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:47,025 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:47,026 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:47,026 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3348a4e2ede73e060506e7ba945c70b5 in 1654ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:47,026 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6efa5ac2e6e42332f71fcc029b1955d7 in 1654ms, sequenceid=13, compaction requested=true 2023-02-09 22:55:47,026 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:47,026 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:47,026 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:47,026 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 22:55:47,026 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 22:55:47,028 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,028 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:47,028 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,028 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:47,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 22:55:47,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b76aebf[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:47,029 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 22:55:47,029 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,030 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,030 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 22:55:47,030 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,030 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:47,030 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 22:55:47,030 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:47,033 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 22:55:47,033 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:47,033 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,033 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 22:55:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:47,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 22:55:47,036 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 22:55:47,036 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:47,036 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:47,036 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 22:55:47,036 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:47,037 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,037 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:47,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:47,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,037 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:47,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:47,038 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,039 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:47,041 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:47,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 22:55:47,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 22:55:47,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:47,041 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:47,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:47,042 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:47,042 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:47,142 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:47,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:47,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:47,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:47,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:55:47,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:47,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:47,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 22:55:47,149 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 22:55:47,149 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:47,149 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 22:55:47,149 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 22:55:47,149 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,149 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:47,151 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:47,151 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:47,151 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:47,151 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:47,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:47,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 22:55:47,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 22:55:47,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 22:55:47,153 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 22:55:47,153 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:47,153 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 22:55:47,153 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:47,153 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. started... 2023-02-09 22:55:47,153 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:47,153 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:47,153 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6efa5ac2e6e42332f71fcc029b1955d7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:47,153 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:47,153 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:47,154 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. started... 2023-02-09 22:55:47,154 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. started... 2023-02-09 22:55:47,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3348a4e2ede73e060506e7ba945c70b5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:47,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4e0b246c234b9e9ec553bf509305303a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:55:47,176 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209718d17f8688b408691c16d57e064823f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230209718d17f8688b408691c16d57e064823f 2023-02-09 22:55:47,573 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020929d6a2d063e8472bb8a0f9f7ddacb90f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023020929d6a2d063e8472bb8a0f9f7ddacb90f 2023-02-09 22:55:47,573 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209a942af96a23541fa9fd8da19bdab9def to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230209a942af96a23541fa9fd8da19bdab9def 2023-02-09 22:55:47,580 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/18893b50a441446d98d31ad78b6d43a2 2023-02-09 22:55:47,978 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/cb69a9f7d08141389240d06542348d3b 2023-02-09 22:55:47,978 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/bca93e697de8465f89318c04c9eea4f9 2023-02-09 22:55:48,000 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209d5383b4c04bd47d8bfea010c4c9bcb64 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230209d5383b4c04bd47d8bfea010c4c9bcb64 2023-02-09 22:55:48,401 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209bcbbfa3af171463392c5726b7d717678 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230209bcbbfa3af171463392c5726b7d717678 2023-02-09 22:55:48,401 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302099ed320b7a596417c8785232cca6d232f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302099ed320b7a596417c8785232cca6d232f 2023-02-09 22:55:48,404 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/534d8d6dc8574d439bd766f40f383991 2023-02-09 22:55:48,407 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/18893b50a441446d98d31ad78b6d43a2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2 2023-02-09 22:55:48,411 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:48,411 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/534d8d6dc8574d439bd766f40f383991 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991 2023-02-09 22:55:48,414 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:48,414 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4e0b246c234b9e9ec553bf509305303a in 1260ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:48,415 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:48,415 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:48,415 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:48,806 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/cdcf5ec296dc43959a2ddce2d48bff49 2023-02-09 22:55:48,806 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/cb4ce09f6fe541ceb41ab0a3193e0aad 2023-02-09 22:55:48,810 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/bca93e697de8465f89318c04c9eea4f9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9 2023-02-09 22:55:48,810 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/cb69a9f7d08141389240d06542348d3b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b 2023-02-09 22:55:48,813 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:48,813 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b, entries=4, sequenceid=17, filesize=5.4 K 2023-02-09 22:55:48,814 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/cb4ce09f6fe541ceb41ab0a3193e0aad as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad 2023-02-09 22:55:48,814 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/cdcf5ec296dc43959a2ddce2d48bff49 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49 2023-02-09 22:55:48,818 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:48,818 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:55:48,818 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3348a4e2ede73e060506e7ba945c70b5 in 1664ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:48,818 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6efa5ac2e6e42332f71fcc029b1955d7 in 1665ms, sequenceid=17, compaction requested=true 2023-02-09 22:55:48,818 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:48,819 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:48,819 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:48,819 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 22:55:48,819 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 22:55:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,820 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,820 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 22:55:48,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@255b1faf[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:48,821 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 22:55:48,822 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,822 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,823 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 22:55:48,823 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 22:55:48,823 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:48,823 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,823 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:48,825 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,825 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:48,825 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:48,825 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 22:55:48,827 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 22:55:48,827 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:48,827 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:48,827 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 22:55:48,827 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:48,828 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,828 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:48,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,828 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:48,829 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:48,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,830 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:48,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:48,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:48,832 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:48,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 22:55:48,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:48,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:48,832 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,833 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:48,833 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:48,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:48,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:48,933 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:48,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:48,940 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:48,947 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:48,954 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:48,955 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:48,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:48,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:48,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-09 22:55:48,979 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-09 22:55:48,979 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:48,979 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-09 22:55:48,979 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-09 22:55:48,979 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,979 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:48,980 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,980 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:48,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:48,981 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:48,981 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:48,981 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-09 22:55:48,981 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-09 22:55:48,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-09 22:55:48,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-09 22:55:48,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:48,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. started... 2023-02-09 22:55:48,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:48,982 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6efa5ac2e6e42332f71fcc029b1955d7 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. started... 2023-02-09 22:55:48,982 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:48,982 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. started... 2023-02-09 22:55:48,982 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 3348a4e2ede73e060506e7ba945c70b5 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:48,982 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4e0b246c234b9e9ec553bf509305303a 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:55:49,798 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/034f6d9e3c8d45e487274579a95ae467 2023-02-09 22:55:49,800 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d 2023-02-09 22:55:49,800 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/298be2ea301b438b9af9a9ae5c7b3d70 2023-02-09 22:55:49,803 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 034f6d9e3c8d45e487274579a95ae467 2023-02-09 22:55:49,808 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 298be2ea301b438b9af9a9ae5c7b3d70 2023-02-09 22:55:49,812 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f2e53ae9ac74382a7a13a79e6e6fe4d 2023-02-09 22:55:50,236 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/9813f5744876420fbfd84fc7ac2738bc 2023-02-09 22:55:50,239 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9813f5744876420fbfd84fc7ac2738bc 2023-02-09 22:55:50,239 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/298be2ea301b438b9af9a9ae5c7b3d70 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70 2023-02-09 22:55:50,242 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 298be2ea301b438b9af9a9ae5c7b3d70 2023-02-09 22:55:50,242 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,243 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/9813f5744876420fbfd84fc7ac2738bc as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc 2023-02-09 22:55:50,246 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9813f5744876420fbfd84fc7ac2738bc 2023-02-09 22:55:50,246 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,247 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4e0b246c234b9e9ec553bf509305303a in 1265ms, sequenceid=23, compaction requested=true 2023-02-09 22:55:50,247 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:50,247 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:50,248 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:55:50,620 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/a2433dae12be488fbf9eddcfc9cafd47 2023-02-09 22:55:50,625 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2433dae12be488fbf9eddcfc9cafd47 2023-02-09 22:55:50,625 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/034f6d9e3c8d45e487274579a95ae467 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467 2023-02-09 22:55:50,629 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 034f6d9e3c8d45e487274579a95ae467 2023-02-09 22:55:50,629 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,629 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/a2433dae12be488fbf9eddcfc9cafd47 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47 2023-02-09 22:55:50,632 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/3a1fff5ede20451d8896c0fc675eb021 2023-02-09 22:55:50,633 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2433dae12be488fbf9eddcfc9cafd47 2023-02-09 22:55:50,633 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,633 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6efa5ac2e6e42332f71fcc029b1955d7 in 1651ms, sequenceid=23, compaction requested=true 2023-02-09 22:55:50,634 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:50,634 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:50,634 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:55:50,635 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a1fff5ede20451d8896c0fc675eb021 2023-02-09 22:55:50,636 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d 2023-02-09 22:55:50,638 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f2e53ae9ac74382a7a13a79e6e6fe4d 2023-02-09 22:55:50,638 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d, entries=3, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,639 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/3a1fff5ede20451d8896c0fc675eb021 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021 2023-02-09 22:55:50,642 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3a1fff5ede20451d8896c0fc675eb021 2023-02-09 22:55:50,642 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:55:50,642 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3348a4e2ede73e060506e7ba945c70b5 in 1660ms, sequenceid=23, compaction requested=true 2023-02-09 22:55:50,642 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:50,642 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:50,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:55:50,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:55:50,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-09 22:55:50,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-09 22:55:50,644 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,644 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,645 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-09 22:55:50,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ba64eed[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:50,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-09 22:55:50,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,648 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-09 22:55:50,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-09 22:55:50,648 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:50,648 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,648 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:50,653 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-09 22:55:50,653 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,653 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,653 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-09 22:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:50,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-09 22:55:50,655 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-09 22:55:50,655 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:50,655 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:50,655 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-09 22:55:50,655 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:50,657 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,657 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:50,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,657 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:50,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,658 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:50,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:50,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,659 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-09 22:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:50,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:50,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-09 22:55:50,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:50,661 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-09 22:55:50,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:50,661 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-09 22:55:50,662 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:50,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-09 22:55:50,762 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:50,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:50,763 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:50,763 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 6efa5ac2e6e42332f71fcc029b1955d7/family1 is initiating major compaction (all files) 2023-02-09 22:55:50,763 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:50,763 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:50,763 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 6efa5ac2e6e42332f71fcc029b1955d7/family1 in ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:50,763 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp, totalSize=26.6 K 2023-02-09 22:55:50,764 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 68634e7e9e6442fa81fd495443ef9c22, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341796 2023-02-09 22:55:50,764 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting d2c2ae2e855f44a690e36276d9ad7806, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343581 2023-02-09 22:55:50,765 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 302268d7ccb04e8995b09e5d0ddcd243, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:50,765 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting bca93e697de8465f89318c04c9eea4f9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:50,766 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 034f6d9e3c8d45e487274579a95ae467, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:50,775 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 6efa5ac2e6e42332f71fcc029b1955d7#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:51,183 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209b2c97706dbb74ad0a9786a64e8ba6af4_del 2023-02-09 22:55:51,183 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209b2c97706dbb74ad0a9786a64e8ba6af4_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209b2c97706dbb74ad0a9786a64e8ba6af4_del 2023-02-09 22:55:51,997 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 94666ddf6b964ef79878697fad4f9d7b 2023-02-09 22:55:51,998 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family1/94666ddf6b964ef79878697fad4f9d7b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/94666ddf6b964ef79878697fad4f9d7b 2023-02-09 22:55:52,002 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 94666ddf6b964ef79878697fad4f9d7b 2023-02-09 22:55:52,003 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6efa5ac2e6e42332f71fcc029b1955d7/family1 of 6efa5ac2e6e42332f71fcc029b1955d7 into 94666ddf6b964ef79878697fad4f9d7b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:52,003 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:52,003 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:52,004 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 6efa5ac2e6e42332f71fcc029b1955d7/family2 is initiating major compaction (all files) 2023-02-09 22:55:52,004 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:52,004 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:52,004 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 6efa5ac2e6e42332f71fcc029b1955d7/family2 in ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:55:52,004 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp, totalSize=25.3 K 2023-02-09 22:55:52,004 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 46aac46fd71742579ce21591d0464e6b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341796 2023-02-09 22:55:52,005 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting f922065d49494886bbf5dc1a49db2cb3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343581 2023-02-09 22:55:52,005 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting bfeb26737f424a13a33b33a9e71d0204, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:52,006 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting cdcf5ec296dc43959a2ddce2d48bff49, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:52,006 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting a2433dae12be488fbf9eddcfc9cafd47, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:52,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467] to archive 2023-02-09 22:55:52,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/68634e7e9e6442fa81fd495443ef9c22 2023-02-09 22:55:52,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d2c2ae2e855f44a690e36276d9ad7806 2023-02-09 22:55:52,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/302268d7ccb04e8995b09e5d0ddcd243 2023-02-09 22:55:52,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/bca93e697de8465f89318c04c9eea4f9 2023-02-09 22:55:52,022 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/034f6d9e3c8d45e487274579a95ae467 2023-02-09 22:55:52,022 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 6efa5ac2e6e42332f71fcc029b1955d7#family2#compaction#151 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:52,433 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302099716a13e252f493a8b3361af55f38b17_del 2023-02-09 22:55:52,433 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302099716a13e252f493a8b3361af55f38b17_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302099716a13e252f493a8b3361af55f38b17_del 2023-02-09 22:55:52,858 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef1667aa5d8e423c9e63b7b01560595b 2023-02-09 22:55:52,859 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/.tmp/family2/ef1667aa5d8e423c9e63b7b01560595b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/ef1667aa5d8e423c9e63b7b01560595b 2023-02-09 22:55:52,862 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ef1667aa5d8e423c9e63b7b01560595b 2023-02-09 22:55:52,864 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6efa5ac2e6e42332f71fcc029b1955d7/family2 of 6efa5ac2e6e42332f71fcc029b1955d7 into ef1667aa5d8e423c9e63b7b01560595b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:52,864 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:52,864 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:52,864 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 4e0b246c234b9e9ec553bf509305303a/family1 is initiating major compaction (all files) 2023-02-09 22:55:52,864 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:52,864 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:52,864 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 4e0b246c234b9e9ec553bf509305303a/family1 in ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:52,864 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp, totalSize=26.6 K 2023-02-09 22:55:52,865 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting c4db2118e0394b25b53ecd3b172d619b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341796 2023-02-09 22:55:52,865 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 4599793e540e4a52b86553dafb33fba4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343581 2023-02-09 22:55:52,865 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 2cdc8207f3c147649eff217f7ad9d3a4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:52,866 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 18893b50a441446d98d31ad78b6d43a2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:52,866 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 298be2ea301b438b9af9a9ae5c7b3d70, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:52,874 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 4e0b246c234b9e9ec553bf509305303a#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:52,889 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020900e7edd0331c43718b0e18568f4ebd13_del 2023-02-09 22:55:52,889 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020900e7edd0331c43718b0e18568f4ebd13_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023020900e7edd0331c43718b0e18568f4ebd13_del 2023-02-09 22:55:52,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47] to archive 2023-02-09 22:55:52,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:52,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/46aac46fd71742579ce21591d0464e6b 2023-02-09 22:55:52,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/f922065d49494886bbf5dc1a49db2cb3 2023-02-09 22:55:52,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/bfeb26737f424a13a33b33a9e71d0204 2023-02-09 22:55:52,920 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/cdcf5ec296dc43959a2ddce2d48bff49 2023-02-09 22:55:52,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family2/a2433dae12be488fbf9eddcfc9cafd47 2023-02-09 22:55:53,304 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b22fd5f9f974171a3a5f13dad7a9568 2023-02-09 22:55:53,305 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family1/5b22fd5f9f974171a3a5f13dad7a9568 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/5b22fd5f9f974171a3a5f13dad7a9568 2023-02-09 22:55:53,309 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b22fd5f9f974171a3a5f13dad7a9568 2023-02-09 22:55:53,310 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4e0b246c234b9e9ec553bf509305303a/family1 of 4e0b246c234b9e9ec553bf509305303a into 5b22fd5f9f974171a3a5f13dad7a9568(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:53,310 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:53,310 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:53,310 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 4e0b246c234b9e9ec553bf509305303a/family2 is initiating major compaction (all files) 2023-02-09 22:55:53,310 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:53,310 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:53,310 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 4e0b246c234b9e9ec553bf509305303a/family2 in ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:55:53,311 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp, totalSize=25.3 K 2023-02-09 22:55:53,311 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 2b55321a268a4f97b12ec7032710b15d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341796 2023-02-09 22:55:53,311 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 67c23179259a4f74a3cd5f42fcf600cb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343581 2023-02-09 22:55:53,311 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 8461986876c64e2183e6a6974ea43555, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:53,312 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 534d8d6dc8574d439bd766f40f383991, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:53,312 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 9813f5744876420fbfd84fc7ac2738bc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:53,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70] to archive 2023-02-09 22:55:53,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:53,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/c4db2118e0394b25b53ecd3b172d619b 2023-02-09 22:55:53,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/4599793e540e4a52b86553dafb33fba4 2023-02-09 22:55:53,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/2cdc8207f3c147649eff217f7ad9d3a4 2023-02-09 22:55:53,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/18893b50a441446d98d31ad78b6d43a2 2023-02-09 22:55:53,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/298be2ea301b438b9af9a9ae5c7b3d70 2023-02-09 22:55:53,321 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 4e0b246c234b9e9ec553bf509305303a#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:53,728 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302094cbcb17369d047e591a862ebe77e2451_del 2023-02-09 22:55:53,728 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302094cbcb17369d047e591a862ebe77e2451_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302094cbcb17369d047e591a862ebe77e2451_del 2023-02-09 22:55:54,548 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00f2734d4b4c4322af6be6c33722f361 2023-02-09 22:55:54,549 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/.tmp/family2/00f2734d4b4c4322af6be6c33722f361 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/00f2734d4b4c4322af6be6c33722f361 2023-02-09 22:55:54,552 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00f2734d4b4c4322af6be6c33722f361 2023-02-09 22:55:54,553 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4e0b246c234b9e9ec553bf509305303a/family2 of 4e0b246c234b9e9ec553bf509305303a into 00f2734d4b4c4322af6be6c33722f361(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:54,553 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:54,553 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:54,553 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 3348a4e2ede73e060506e7ba945c70b5/family1 is initiating major compaction (all files) 2023-02-09 22:55:54,553 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:54,554 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:54,554 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 3348a4e2ede73e060506e7ba945c70b5/family1 in ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:54,554 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp, totalSize=26.6 K 2023-02-09 22:55:54,554 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting bdb8f24621424d3c9b7c46c558390009, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341795 2023-02-09 22:55:54,554 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting c03092e651ea46b8bd3ecef141931333, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343580 2023-02-09 22:55:54,555 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 36ee1a2efa6142ef9293d8861f8afbf0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:54,555 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting cb69a9f7d08141389240d06542348d3b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:54,555 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 5f2e53ae9ac74382a7a13a79e6e6fe4d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:54,564 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 3348a4e2ede73e060506e7ba945c70b5#family1#compaction#154 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:54,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc] to archive 2023-02-09 22:55:54,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:54,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/2b55321a268a4f97b12ec7032710b15d 2023-02-09 22:55:54,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/67c23179259a4f74a3cd5f42fcf600cb 2023-02-09 22:55:54,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/8461986876c64e2183e6a6974ea43555 2023-02-09 22:55:54,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/534d8d6dc8574d439bd766f40f383991 2023-02-09 22:55:54,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family2/9813f5744876420fbfd84fc7ac2738bc 2023-02-09 22:55:54,972 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302096443b5c59ed346ef9b6c8bad56a5bed6_del 2023-02-09 22:55:54,972 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302096443b5c59ed346ef9b6c8bad56a5bed6_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302096443b5c59ed346ef9b6c8bad56a5bed6_del 2023-02-09 22:55:55,788 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 89d42f51ab6d4eb597cfa549da69c52c 2023-02-09 22:55:55,789 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family1/89d42f51ab6d4eb597cfa549da69c52c as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/89d42f51ab6d4eb597cfa549da69c52c 2023-02-09 22:55:55,794 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 89d42f51ab6d4eb597cfa549da69c52c 2023-02-09 22:55:55,795 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3348a4e2ede73e060506e7ba945c70b5/family1 of 3348a4e2ede73e060506e7ba945c70b5 into 89d42f51ab6d4eb597cfa549da69c52c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:55:55,795 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:55,795 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:55:55,796 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 3348a4e2ede73e060506e7ba945c70b5/family2 is initiating major compaction (all files) 2023-02-09 22:55:55,796 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:55:55,796 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:55:55,798 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 3348a4e2ede73e060506e7ba945c70b5/family2 in ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:55:55,798 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp, totalSize=25.3 K 2023-02-09 22:55:55,799 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 8a87e25671a846518bc611700695b749, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983341795 2023-02-09 22:55:55,799 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 676152435ae14718a010cd5474049306, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983343580 2023-02-09 22:55:55,800 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 6e92e7d7dd1e4dae98c2eeb8645f0616, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983345365 2023-02-09 22:55:55,800 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting cb4ce09f6fe541ceb41ab0a3193e0aad, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983347144 2023-02-09 22:55:55,800 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 3a1fff5ede20451d8896c0fc675eb021, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:55:55,810 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 3348a4e2ede73e060506e7ba945c70b5#family2#compaction#155 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:55:55,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d] to archive 2023-02-09 22:55:55,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:55,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/bdb8f24621424d3c9b7c46c558390009 2023-02-09 22:55:55,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/c03092e651ea46b8bd3ecef141931333 2023-02-09 22:55:55,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/36ee1a2efa6142ef9293d8861f8afbf0 2023-02-09 22:55:55,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/cb69a9f7d08141389240d06542348d3b 2023-02-09 22:55:55,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/5f2e53ae9ac74382a7a13a79e6e6fe4d 2023-02-09 22:55:56,222 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023020942d781772905427999f807d7d59e2899_del 2023-02-09 22:55:56,223 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020942d781772905427999f807d7d59e2899_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020942d781772905427999f807d7d59e2899_del 2023-02-09 22:55:56,639 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14eacff3f37145f5aaacf319909b5d79 2023-02-09 22:55:56,643 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/.tmp/family2/14eacff3f37145f5aaacf319909b5d79 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/14eacff3f37145f5aaacf319909b5d79 2023-02-09 22:55:56,648 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14eacff3f37145f5aaacf319909b5d79 2023-02-09 22:55:56,650 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3348a4e2ede73e060506e7ba945c70b5/family2 of 3348a4e2ede73e060506e7ba945c70b5 into 14eacff3f37145f5aaacf319909b5d79(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:55:56,650 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:56,654 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:56,657 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:56,661 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:56,661 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:56,667 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:55:56,682 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-09 22:55:56,684 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302096443b5c59ed346ef9b6c8bad56a5bed6_del 2023-02-09 22:55:56,687 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230209b2c97706dbb74ad0a9786a64e8ba6af4_del 2023-02-09 22:55:56,690 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020900e7edd0331c43718b0e18568f4ebd13_del 2023-02-09 22:55:56,690 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:56,690 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(139): Connect 0x3215ecc8 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:55:56,696 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@50eba383, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:55:56,697 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:56,697 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:56,697 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:56,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021] to archive 2023-02-09 22:55:56,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:56,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/8a87e25671a846518bc611700695b749 2023-02-09 22:55:56,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/676152435ae14718a010cd5474049306 2023-02-09 22:55:56,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/6e92e7d7dd1e4dae98c2eeb8645f0616 2023-02-09 22:55:56,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/cb4ce09f6fe541ceb41ab0a3193e0aad 2023-02-09 22:55:56,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family2/3a1fff5ede20451d8896c0fc675eb021 2023-02-09 22:55:56,764 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020982b38ae508f448a8afe8da19f0754f78 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023020982b38ae508f448a8afe8da19f0754f78 2023-02-09 22:55:56,765 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:56,783 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:55:56,786 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:58988, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:55:56,798 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:55:56,801 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:40012, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:55:56,814 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/2082b98dbebc4c2f9a70efa84f4c3852 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-09 22:55:56,814 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/2082b98dbebc4c2f9a70efa84f4c3852}] 2023-02-09 22:55:56,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/2082b98dbebc4c2f9a70efa84f4c3852 for inclusion in 4e0b246c234b9e9ec553bf509305303a/family1 2023-02-09 22:55:56,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-09 22:55:56,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 22:55:56,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 4e0b246c234b9e9ec553bf509305303a: 2023-02-09 22:55:56,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209/family1/2082b98dbebc4c2f9a70efa84f4c3852 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__et6auhk73f4e06et2o8hr9m9b8g95qpjd3tmfb4a17nl3ajaeb7i5a757e8rgsro/family1/2082b98dbebc4c2f9a70efa84f4c3852 2023-02-09 22:55:56,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__et6auhk73f4e06et2o8hr9m9b8g95qpjd3tmfb4a17nl3ajaeb7i5a757e8rgsro/family1/2082b98dbebc4c2f9a70efa84f4c3852 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/70b7b412b63241a48a033cde0e92e46f_SeqId_28_ 2023-02-09 22:55:56,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__et6auhk73f4e06et2o8hr9m9b8g95qpjd3tmfb4a17nl3ajaeb7i5a757e8rgsro/family1/2082b98dbebc4c2f9a70efa84f4c3852 into 4e0b246c234b9e9ec553bf509305303a/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/70b7b412b63241a48a033cde0e92e46f_SeqId_28_ - updating store file list. 2023-02-09 22:55:56,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/70b7b412b63241a48a033cde0e92e46f_SeqId_28_ into 4e0b246c234b9e9ec553bf509305303a/family1 2023-02-09 22:55:56,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__et6auhk73f4e06et2o8hr9m9b8g95qpjd3tmfb4a17nl3ajaeb7i5a757e8rgsro/family1/2082b98dbebc4c2f9a70efa84f4c3852 into 4e0b246c234b9e9ec553bf509305303a/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/family1/70b7b412b63241a48a033cde0e92e46f_SeqId_28_) 2023-02-09 22:55:56,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__et6auhk73f4e06et2o8hr9m9b8g95qpjd3tmfb4a17nl3ajaeb7i5a757e8rgsro/family1/2082b98dbebc4c2f9a70efa84f4c3852 2023-02-09 22:55:56,831 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:56,832 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:56,833 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302093c60cf23522d40c985bda6b8ba71b35b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302093c60cf23522d40c985bda6b8ba71b35b 2023-02-09 22:55:56,833 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 22:55:56,833 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 22:55:56,834 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:57,163 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302094289f4a42290492388687d8ab9847fb0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302094289f4a42290492388687d8ab9847fb0 2023-02-09 22:55:57,164 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:57,194 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/e53bbbe6a82d4383a2b1f32e9164eb8e first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-09 22:55:57,195 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/e53bbbe6a82d4383a2b1f32e9164eb8e}] 2023-02-09 22:55:57,199 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/e53bbbe6a82d4383a2b1f32e9164eb8e for inclusion in 6efa5ac2e6e42332f71fcc029b1955d7/family1 2023-02-09 22:55:57,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-09 22:55:57,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 22:55:57,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 6efa5ac2e6e42332f71fcc029b1955d7: 2023-02-09 22:55:57,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209/family1/e53bbbe6a82d4383a2b1f32e9164eb8e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__qg187plr44g4gnv2hf1oaqm4h0517dr4cbdtenkllddr0t5447pma5cokrdr2i0i/family1/e53bbbe6a82d4383a2b1f32e9164eb8e 2023-02-09 22:55:57,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__qg187plr44g4gnv2hf1oaqm4h0517dr4cbdtenkllddr0t5447pma5cokrdr2i0i/family1/e53bbbe6a82d4383a2b1f32e9164eb8e as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d419458b166d43d389795e91e1a79134_SeqId_28_ 2023-02-09 22:55:57,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__qg187plr44g4gnv2hf1oaqm4h0517dr4cbdtenkllddr0t5447pma5cokrdr2i0i/family1/e53bbbe6a82d4383a2b1f32e9164eb8e into 6efa5ac2e6e42332f71fcc029b1955d7/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d419458b166d43d389795e91e1a79134_SeqId_28_ - updating store file list. 2023-02-09 22:55:57,210 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d419458b166d43d389795e91e1a79134_SeqId_28_ into 6efa5ac2e6e42332f71fcc029b1955d7/family1 2023-02-09 22:55:57,210 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__qg187plr44g4gnv2hf1oaqm4h0517dr4cbdtenkllddr0t5447pma5cokrdr2i0i/family1/e53bbbe6a82d4383a2b1f32e9164eb8e into 6efa5ac2e6e42332f71fcc029b1955d7/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/family1/d419458b166d43d389795e91e1a79134_SeqId_28_) 2023-02-09 22:55:57,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__qg187plr44g4gnv2hf1oaqm4h0517dr4cbdtenkllddr0t5447pma5cokrdr2i0i/family1/e53bbbe6a82d4383a2b1f32e9164eb8e 2023-02-09 22:55:57,213 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:57,213 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:57,215 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209442bc4feec594618ba869b59a7cc6e29 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230209442bc4feec594618ba869b59a7cc6e29 2023-02-09 22:55:57,215 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 22:55:57,215 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 22:55:57,215 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:57,557 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302098d6d46b3a86041ea88eaba5f127bfb18 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302098d6d46b3a86041ea88eaba5f127bfb18 2023-02-09 22:55:57,558 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:57,583 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/ac7b72adc7e841288460a95946e6f659 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-09 22:55:57,584 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/ac7b72adc7e841288460a95946e6f659}] 2023-02-09 22:55:57,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/ac7b72adc7e841288460a95946e6f659 for inclusion in 3348a4e2ede73e060506e7ba945c70b5/family1 2023-02-09 22:55:57,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-09 22:55:57,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 22:55:57,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 3348a4e2ede73e060506e7ba945c70b5: 2023-02-09 22:55:57,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209/family1/ac7b72adc7e841288460a95946e6f659 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__o7ri5r0jgvd2scc5ca91p1v3h7di9jcq8ubope4rbhvvb8sb5473iv0ngt28ossk/family1/ac7b72adc7e841288460a95946e6f659 2023-02-09 22:55:57,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__o7ri5r0jgvd2scc5ca91p1v3h7di9jcq8ubope4rbhvvb8sb5473iv0ngt28ossk/family1/ac7b72adc7e841288460a95946e6f659 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/9cecc4b033414b7fa4d16de1684721c3_SeqId_28_ 2023-02-09 22:55:57,591 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__o7ri5r0jgvd2scc5ca91p1v3h7di9jcq8ubope4rbhvvb8sb5473iv0ngt28ossk/family1/ac7b72adc7e841288460a95946e6f659 into 3348a4e2ede73e060506e7ba945c70b5/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/9cecc4b033414b7fa4d16de1684721c3_SeqId_28_ - updating store file list. 2023-02-09 22:55:57,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/9cecc4b033414b7fa4d16de1684721c3_SeqId_28_ into 3348a4e2ede73e060506e7ba945c70b5/family1 2023-02-09 22:55:57,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__o7ri5r0jgvd2scc5ca91p1v3h7di9jcq8ubope4rbhvvb8sb5473iv0ngt28ossk/family1/ac7b72adc7e841288460a95946e6f659 into 3348a4e2ede73e060506e7ba945c70b5/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/family1/9cecc4b033414b7fa4d16de1684721c3_SeqId_28_) 2023-02-09 22:55:57,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__ns_testMinorCompaction__o7ri5r0jgvd2scc5ca91p1v3h7di9jcq8ubope4rbhvvb8sb5473iv0ngt28ossk/family1/ac7b72adc7e841288460a95946e6f659 2023-02-09 22:55:57,596 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:57,596 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:55:57,598 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302092889d5e3e393497abf687380e38a5baa to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302092889d5e3e393497abf687380e38a5baa 2023-02-09 22:55:57,598 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-09 22:55:57,598 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 22:55:57,598 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='ns:testMinorCompaction' and column='family1' 2023-02-09 22:55:57,598 INFO [Listener at localhost/33819] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-09 22:55:57,600 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:57,602 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:57,604 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:55:57,604 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:55:57,622 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=250 (was 283), OpenFileDescriptor=629 (was 755), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=154 (was 104) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=11247 (was 10664) - AvailableMemoryMB LEAK? - 2023-02-09 22:55:57,632 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=250, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=154, ProcessCount=168, AvailableMemoryMB=11246 2023-02-09 22:55:57,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-09 22:55:57,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:57,635 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:55:57,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-09 22:55:57,636 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:55:57,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 22:55:57,637 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:57,637 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963 empty. 2023-02-09 22:55:57,638 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:57,638 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-09 22:55:57,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 22:55:57,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 22:55:58,045 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-09 22:55:58,046 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 662ddac1568b4f798ff296d3ce34f963, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:55:58,051 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:58,051 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 662ddac1568b4f798ff296d3ce34f963, disabling compactions & flushes 2023-02-09 22:55:58,051 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,051 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,051 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. after waiting 0 ms 2023-02-09 22:55:58,051 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,051 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,052 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 662ddac1568b4f798ff296d3ce34f963: Waiting for close lock at 1675983358051Disabling compacts and flushes for region at 1675983358051Disabling writes for close at 1675983358051Writing region close event to WAL at 1675983358051Closed at 1675983358051 2023-02-09 22:55:58,053 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:55:58,054 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983358054"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983358054"}]},"ts":"1675983358054"} 2023-02-09 22:55:58,056 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 22:55:58,056 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:55:58,057 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983358056"}]},"ts":"1675983358056"} 2023-02-09 22:55:58,057 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-09 22:55:58,066 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=662ddac1568b4f798ff296d3ce34f963, ASSIGN}] 2023-02-09 22:55:58,068 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=662ddac1568b4f798ff296d3ce34f963, ASSIGN 2023-02-09 22:55:58,068 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=662ddac1568b4f798ff296d3ce34f963, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:55:58,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=662ddac1568b4f798ff296d3ce34f963, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:58,220 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983358219"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983358219"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983358219"}]},"ts":"1675983358219"} 2023-02-09 22:55:58,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 662ddac1568b4f798ff296d3ce34f963, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:55:58,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 22:55:58,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 662ddac1568b4f798ff296d3ce34f963, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.', STARTKEY => '', ENDKEY => ''} 2023-02-09 22:55:58,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:55:58,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,378 INFO [StoreOpener-662ddac1568b4f798ff296d3ce34f963-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,379 DEBUG [StoreOpener-662ddac1568b4f798ff296d3ce34f963-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:55:58,379 DEBUG [StoreOpener-662ddac1568b4f798ff296d3ce34f963-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:55:58,379 INFO [StoreOpener-662ddac1568b4f798ff296d3ce34f963-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 662ddac1568b4f798ff296d3ce34f963 columnFamilyName family1 2023-02-09 22:55:58,381 INFO [StoreOpener-662ddac1568b4f798ff296d3ce34f963-1] regionserver.HStore(310): Store=662ddac1568b4f798ff296d3ce34f963/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:55:58,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:55:58,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:55:58,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 662ddac1568b4f798ff296d3ce34f963; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63000864, jitterRate=-0.06121397018432617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 22:55:58,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 662ddac1568b4f798ff296d3ce34f963: Running coprocessor pre-open hook at 1675983358377Writing region info on filesystem at 1675983358377Initializing all the Stores at 1675983358377Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675983358377Cleaning up temporary data from old regions at 1675983358382 (+5 ms)Cleaning up detritus from prior splits at 1675983358383 (+1 ms)Running coprocessor post-open hooks at 1675983358387 (+4 ms)Region opened successfully at 1675983358387 2023-02-09 22:55:58,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963., pid=114, masterSystemTime=1675983358373 2023-02-09 22:55:58,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,389 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=662ddac1568b4f798ff296d3ce34f963, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:58,389 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675983358389"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983358389"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983358389"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983358389"}]},"ts":"1675983358389"} 2023-02-09 22:55:58,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 22:55:58,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 662ddac1568b4f798ff296d3ce34f963, server=jenkins-hbase9.apache.org,39093,1675983215026 in 170 msec 2023-02-09 22:55:58,393 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 22:55:58,393 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=662ddac1568b4f798ff296d3ce34f963, ASSIGN in 326 msec 2023-02-09 22:55:58,393 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:55:58,394 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983358394"}]},"ts":"1675983358394"} 2023-02-09 22:55:58,395 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-09 22:55:58,396 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:55:58,397 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 762 msec 2023-02-09 22:55:58,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-09 22:55:58,741 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-09 22:55:58,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:58,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,769 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:55:58,769 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:58,769 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:55:58,769 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:55:58,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,770 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,771 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:58,771 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:58,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:58,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:58,771 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:58,771 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:58,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,772 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:58,777 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:55:58,777 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:58,777 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:55:58,777 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:58,777 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:58,777 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:58,777 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:55:58,778 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-09 22:55:58,802 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e9ae63c27b894543abc5e5dde5a4c882 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e9ae63c27b894543abc5e5dde5a4c882 2023-02-09 22:55:59,207 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/66ebaafbe18943ccb39c1b2ea8113a05 2023-02-09 22:55:59,211 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/66ebaafbe18943ccb39c1b2ea8113a05 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/66ebaafbe18943ccb39c1b2ea8113a05 2023-02-09 22:55:59,214 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/66ebaafbe18943ccb39c1b2ea8113a05, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-09 22:55:59,215 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 437ms, sequenceid=5, compaction requested=false 2023-02-09 22:55:59,215 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:55:59,215 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:55:59,215 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:59,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:55:59,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:55:59,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:55:59,215 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:55:59,218 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,218 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:59,219 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,219 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:59,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:55:59,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59f3228b[Count = 0] remaining members to acquire global barrier 2023-02-09 22:55:59,220 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:55:59,220 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,221 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,221 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,222 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:55:59,221 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:55:59,222 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:55:59,222 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:55:59,223 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,223 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:55:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:59,223 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:59,223 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:55:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:55:59,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:55:59,225 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:55:59,225 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:55:59,225 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:55:59,225 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,225 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:55:59,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,227 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:55:59,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:55:59,227 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:55:59,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,228 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:55:59,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,231 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:59,231 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:59,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:59,231 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:55:59,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:55:59,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:55:59,231 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:55:59,231 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:55:59,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:55:59,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,232 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,332 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:55:59,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:55:59,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:55:59,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:55:59,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,338 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:55:59,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:59,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:55:59,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:55:59,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,338 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,339 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,339 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:55:59,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:55:59,340 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:55:59,340 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:55:59,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:55:59,341 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:55:59,341 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:55:59,341 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:55:59,341 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:55:59,341 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:55:59,341 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:55:59,341 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:55:59,341 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-09 22:55:59,753 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283bde111c768743ff804c93a5513e437b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283bde111c768743ff804c93a5513e437b 2023-02-09 22:56:00,158 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1adf88689613483fb5db7f2d9ca370ab 2023-02-09 22:56:00,161 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1adf88689613483fb5db7f2d9ca370ab as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab 2023-02-09 22:56:00,165 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab, entries=1, sequenceid=9, filesize=5.0 K 2023-02-09 22:56:00,165 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 824ms, sequenceid=9, compaction requested=false 2023-02-09 22:56:00,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:00,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:00,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:00,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:00,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:00,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:00,168 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,168 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:00,168 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:00,168 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:00,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:00,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:00,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78358948[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:00,169 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:00,169 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,170 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,170 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:00,170 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:00,170 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,170 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:00,171 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:00,178 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,178 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:00,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,178 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:00,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:00,178 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:00,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:00,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:00,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:00,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:00,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:00,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:00,180 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:00,180 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:00,180 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:00,180 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,180 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:00,182 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,182 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,182 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:00,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:00,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:00,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:00,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:00,182 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,183 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:00,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:00,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:00,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:00,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:00,188 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:00,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:00,188 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:00,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:00,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:00,288 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:00,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:00,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:00,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:00,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,294 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:00,294 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:00,294 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:00,294 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:00,294 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,294 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,301 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,301 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:00,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:00,302 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:00,302 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:00,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:00,303 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:00,303 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:00,303 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:00,303 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:00,303 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:00,303 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:00,303 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:00,303 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:00,714 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120188f6ee6f9f654755a7c6b1de17ec391a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120188f6ee6f9f654755a7c6b1de17ec391a 2023-02-09 22:56:01,119 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1839ff985f6d40a69af736764e2a1884 2023-02-09 22:56:01,122 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1839ff985f6d40a69af736764e2a1884 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884 2023-02-09 22:56:01,126 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884, entries=1, sequenceid=13, filesize=4.9 K 2023-02-09 22:56:01,127 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 824ms, sequenceid=13, compaction requested=true 2023-02-09 22:56:01,127 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:01,127 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:01,127 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:01,127 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:01,127 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,127 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:01,127 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:01,129 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,129 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:01,129 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,129 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:01,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:01,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56e5786[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:01,130 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:01,130 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,132 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,132 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:01,132 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:01,132 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:01,132 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,132 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:01,133 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,133 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:01,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:01,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:01,133 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:01,134 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:01,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:01,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:01,137 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:01,137 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:01,137 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:01,137 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,137 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:01,140 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,140 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:01,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,140 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:01,140 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:01,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,141 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:01,143 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:01,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:01,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,143 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:01,144 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:01,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:01,144 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:01,244 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:01,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:01,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:01,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:01,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,250 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:01,250 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:01,250 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:01,250 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:01,250 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,250 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,252 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:01,252 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:01,252 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:01,252 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:01,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:01,253 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:01,253 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:01,253 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:01,254 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:01,254 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:01,254 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:01,254 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:01,668 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120553f4d0b2c84f4b00b07cc6dca8eeaec7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120553f4d0b2c84f4b00b07cc6dca8eeaec7 2023-02-09 22:56:02,073 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1682e7f63e3347e7bc2da327277fa682 2023-02-09 22:56:02,078 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/1682e7f63e3347e7bc2da327277fa682 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682 2023-02-09 22:56:02,082 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682, entries=1, sequenceid=17, filesize=4.9 K 2023-02-09 22:56:02,083 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 829ms, sequenceid=17, compaction requested=true 2023-02-09 22:56:02,083 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:02,083 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:02,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:02,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:02,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:02,083 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:02,084 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,084 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,085 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,085 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:02,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c56a0e0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:02,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:02,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,088 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:02,089 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,089 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:02,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:02,089 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:02,090 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:02,090 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,090 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,090 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:02,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:02,092 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:02,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:02,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:02,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,092 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:02,094 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,094 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,094 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:02,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,095 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,095 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,098 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:02,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:02,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:02,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:02,098 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:02,198 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:02,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:02,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:02,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:02,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,205 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:02,205 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:02,205 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:02,205 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:02,205 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,205 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,207 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:02,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:02,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:02,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,208 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,208 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:02,208 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:02,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:02,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:02,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:02,209 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:02,209 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:02,209 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:02,209 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:02,620 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228b16f775bde2647bf8f3206842591a21e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228b16f775bde2647bf8f3206842591a21e 2023-02-09 22:56:02,625 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/c32bc8169f464428b05c595053b1ecfd 2023-02-09 22:56:02,628 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/c32bc8169f464428b05c595053b1ecfd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd 2023-02-09 22:56:02,632 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd, entries=1, sequenceid=21, filesize=4.9 K 2023-02-09 22:56:02,633 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 424ms, sequenceid=21, compaction requested=true 2023-02-09 22:56:02,633 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:02,633 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:02,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:02,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:02,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:02,633 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:02,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,635 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,635 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c71209b[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:02,637 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:02,637 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,638 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:02,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:02,638 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,638 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:02,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:02,640 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:02,640 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:02,640 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,640 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:02,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:02,642 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:02,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:02,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:02,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,642 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:02,643 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,643 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:02,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,644 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:02,644 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:02,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:02,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:02,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:02,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:02,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:02,650 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:02,650 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:02,651 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:02,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:02,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:02,751 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:02,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:02,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:02,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:02,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,757 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:02,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:02,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:02,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:02,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,758 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,758 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:02,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:02,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:02,759 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:02,759 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:02,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:02,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:02,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:02,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:02,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:02,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:02,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:02,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:02,760 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:02,760 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:02,760 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:03,170 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231174401d29629463ea15228d5abe260fd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231174401d29629463ea15228d5abe260fd 2023-02-09 22:56:03,575 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/b205dd1690564a39bf70d1131e8394cc 2023-02-09 22:56:03,579 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/b205dd1690564a39bf70d1131e8394cc as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc 2023-02-09 22:56:03,582 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc, entries=1, sequenceid=25, filesize=4.9 K 2023-02-09 22:56:03,583 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 823ms, sequenceid=25, compaction requested=true 2023-02-09 22:56:03,583 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:03,583 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:03,583 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:03,583 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:03,583 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,583 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:03,583 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:03,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,585 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:03,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,585 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:03,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:03,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:03,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@785a9394[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:03,586 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:03,586 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,588 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:03,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:03,588 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:03,588 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,588 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:03,590 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:03,590 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,590 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,590 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:03,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:03,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:03,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:03,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:03,592 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:03,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:03,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:03,592 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,592 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:03,594 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,594 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,594 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:03,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:03,594 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:03,595 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:03,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,598 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:03,597 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:03,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:03,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:03,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:03,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:03,598 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:03,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:03,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:03,698 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:03,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:03,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:03,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:03,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,704 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:03,704 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:03,705 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:03,705 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:03,705 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,705 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,707 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,707 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:03,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:03,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:03,707 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:03,707 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:03,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,708 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:03,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,708 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:03,708 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:03,708 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:03,708 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:03,709 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:03,709 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:03,709 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:03,709 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:03,709 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:03,709 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:03,914 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:56:04,121 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101fc67b5661ee64003850c09ff92a9ad3d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fc67b5661ee64003850c09ff92a9ad3d 2023-02-09 22:56:04,525 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/dde0eb45693540d19b0e2a68f62ec3e1 2023-02-09 22:56:04,529 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/dde0eb45693540d19b0e2a68f62ec3e1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1 2023-02-09 22:56:04,532 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1, entries=1, sequenceid=29, filesize=4.9 K 2023-02-09 22:56:04,533 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 824ms, sequenceid=29, compaction requested=true 2023-02-09 22:56:04,533 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:04,533 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:04,533 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:04,533 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:04,533 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,533 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:04,533 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:04,535 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,535 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:04,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:04,535 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,535 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:04,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:04,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:04,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:04,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:04,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fff297e[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:04,536 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:04,536 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,538 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,538 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:04,538 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:04,538 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:04,538 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,538 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:04,539 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:04,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,539 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:04,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:04,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:04,539 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:04,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:04,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:04,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:04,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:04,541 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:04,541 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:04,541 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:04,541 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:04,541 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,541 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:04,543 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,543 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,544 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:04,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:04,544 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:04,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:04,545 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:04,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,547 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:04,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:04,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:04,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:04,547 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,548 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,548 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:04,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:04,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:04,548 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:04,648 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:04,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:04,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:04,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:04,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,654 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:04,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:04,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:04,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:04,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,655 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:04,655 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:04,656 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:04,656 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:04,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:04,657 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:04,657 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:04,657 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:04,657 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:04,657 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:04,657 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:04,657 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:04,657 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:04,667 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b2881af6914d4d3cab053a5739abed55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103b2881af6914d4d3cab053a5739abed55 2023-02-09 22:56:05,072 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/68ce072936e44a56ad816d321f405e22 2023-02-09 22:56:05,076 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/68ce072936e44a56ad816d321f405e22 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22 2023-02-09 22:56:05,079 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22, entries=1, sequenceid=33, filesize=4.9 K 2023-02-09 22:56:05,080 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 423ms, sequenceid=33, compaction requested=true 2023-02-09 22:56:05,080 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:05,080 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:05,080 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:05,080 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:05,080 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,080 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:05,080 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:05,082 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,082 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,082 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,082 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:05,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56f7d9c1[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:05,083 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:05,084 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,086 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:05,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:05,086 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:05,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,086 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:05,088 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,088 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,088 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:05,088 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:05,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:05,090 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:05,090 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:05,090 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:05,090 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,090 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:05,092 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,092 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,092 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:05,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,092 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,093 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,099 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,100 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:05,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:05,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:05,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:05,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:05,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:05,100 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,200 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:05,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:05,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:05,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:05,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,207 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:05,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:05,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:05,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:05,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,207 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,208 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:05,208 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:05,209 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,209 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:05,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:05,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:05,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:05,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:05,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:05,210 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:05,210 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:05,210 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:05,210 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:05,223 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070fd5b58266f8457493eec028e46802a9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509070fd5b58266f8457493eec028e46802a9 2023-02-09 22:56:05,628 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/8a3e5002ffa04344a5e2c2ba4593229d 2023-02-09 22:56:05,632 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/8a3e5002ffa04344a5e2c2ba4593229d as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d 2023-02-09 22:56:05,635 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d, entries=1, sequenceid=37, filesize=4.9 K 2023-02-09 22:56:05,636 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 426ms, sequenceid=37, compaction requested=true 2023-02-09 22:56:05,636 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:05,636 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:05,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:05,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:05,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:05,636 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:05,638 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,638 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:05,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73275e7c[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:05,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:05,639 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,641 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:05,641 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:05,642 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:05,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,642 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:05,643 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:05,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:05,643 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:05,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:05,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:05,645 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:05,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:05,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:05,645 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,645 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:05,647 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,647 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,647 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:05,647 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:05,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:05,648 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:05,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:05,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:05,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:05,650 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:05,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:05,650 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,651 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:05,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:05,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:05,651 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:05,651 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:05,651 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,651 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,751 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:05,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:05,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:05,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:05,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,756 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:05,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:05,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-09 22:56:05,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-09 22:56:05,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,757 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,759 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:05,759 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:05,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:05,759 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:05,760 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,760 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-09 22:56:05,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:05,760 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-09 22:56:05,761 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:05,761 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-09 22:56:05,761 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 22:56:05,761 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:05,761 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:05,761 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. started... 2023-02-09 22:56:05,761 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 662ddac1568b4f798ff296d3ce34f963 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-09 22:56:06,172 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511205a0dde6ee719419ca3c796f7a42d4d01 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205a0dde6ee719419ca3c796f7a42d4d01 2023-02-09 22:56:06,179 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/c702758d31ed49fe899688ba1309a232 2023-02-09 22:56:06,183 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/c702758d31ed49fe899688ba1309a232 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232 2023-02-09 22:56:06,186 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232, entries=1, sequenceid=41, filesize=4.9 K 2023-02-09 22:56:06,187 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 662ddac1568b4f798ff296d3ce34f963 in 426ms, sequenceid=41, compaction requested=true 2023-02-09 22:56:06,187 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:06,187 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:06,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 22:56:06,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 22:56:06,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-09 22:56:06,187 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-09 22:56:06,189 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,189 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:06,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:06,189 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,189 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:06,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:06,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:06,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:06,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-09 22:56:06,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f4db5bf[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:06,190 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-09 22:56:06,190 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,192 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,192 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,192 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:06,192 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-09 22:56:06,192 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-09 22:56:06,192 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:06,193 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-09 22:56:06,193 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:06,193 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,193 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-09 22:56:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:06,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:06,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-09 22:56:06,195 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-09 22:56:06,196 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:06,196 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:06,196 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,196 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:06,198 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,198 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:06,198 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,198 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:06,199 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,201 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:06,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-09 22:56:06,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:06,202 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:06,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:06,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-09 22:56:06,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,202 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,303 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:06,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:06,305 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-09 22:56:06,306 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:56:06,307 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:56:06,307 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:56:06,307 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983966307 2023-02-09 22:56:06,309 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-09 22:56:06,310 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-09 22:56:06,310 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:06,310 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x670803c7 to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:56:06,314 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41f316f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:56:06,314 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,139 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cd9b0da9507f4823bfb72df371514511 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103cd9b0da9507f4823bfb72df371514511 2023-02-09 22:56:07,141 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,153 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:56:07,156 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:41148, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:56:07,161 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:56:07,162 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:50752, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:56:07,172 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/900de383f5e342f6ab7644fedacf67a7 first=Optional[r5] last=Optional[r6] 2023-02-09 22:56:07,172 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/900de383f5e342f6ab7644fedacf67a7}] 2023-02-09 22:56:07,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/900de383f5e342f6ab7644fedacf67a7 for inclusion in 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-09 22:56:07,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:56:07,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:07,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/900de383f5e342f6ab7644fedacf67a7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c8bkn0do377fs9skj89dekkkig953pn5kb6di5ju8rvc8abm8oodn1b80jmdsphb/family1/900de383f5e342f6ab7644fedacf67a7 2023-02-09 22:56:07,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c8bkn0do377fs9skj89dekkkig953pn5kb6di5ju8rvc8abm8oodn1b80jmdsphb/family1/900de383f5e342f6ab7644fedacf67a7 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_ 2023-02-09 22:56:07,182 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c8bkn0do377fs9skj89dekkkig953pn5kb6di5ju8rvc8abm8oodn1b80jmdsphb/family1/900de383f5e342f6ab7644fedacf67a7 into 662ddac1568b4f798ff296d3ce34f963/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_ - updating store file list. 2023-02-09 22:56:07,186 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_ into 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,186 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c8bkn0do377fs9skj89dekkkig953pn5kb6di5ju8rvc8abm8oodn1b80jmdsphb/family1/900de383f5e342f6ab7644fedacf67a7 into 662ddac1568b4f798ff296d3ce34f963/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_) 2023-02-09 22:56:07,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c8bkn0do377fs9skj89dekkkig953pn5kb6di5ju8rvc8abm8oodn1b80jmdsphb/family1/900de383f5e342f6ab7644fedacf67a7 2023-02-09 22:56:07,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-09 22:56:07,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-09 22:56:07,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(1912): 662ddac1568b4f798ff296d3ce34f963/family1 is initiating minor compaction 2023-02-09 22:56:07,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7fe005dd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:56:07,189 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 662ddac1568b4f798ff296d3ce34f963/family1 in testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:07,189 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp, totalSize=49.7 K 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 1adf88689613483fb5db7f2d9ca370ab, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 1839ff985f6d40a69af736764e2a1884, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 1682e7f63e3347e7bc2da327277fa682, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c32bc8169f464428b05c595053b1ecfd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting b205dd1690564a39bf70d1131e8394cc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting dde0eb45693540d19b0e2a68f62ec3e1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 68ce072936e44a56ad816d321f405e22, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-09 22:56:07,189 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 8a3e5002ffa04344a5e2c2ba4593229d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-09 22:56:07,190 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting c702758d31ed49fe899688ba1309a232, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-09 22:56:07,190 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] compactions.Compactor(207): Compacting 35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-09 22:56:07,191 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,191 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:07,193 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fc67b5661ee64003850c09ff92a9ad3d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101fc67b5661ee64003850c09ff92a9ad3d 2023-02-09 22:56:07,194 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103b2881af6914d4d3cab053a5739abed55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103b2881af6914d4d3cab053a5739abed55 2023-02-09 22:56:07,194 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-09 22:56:07,194 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-09 22:56:07,194 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,224 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] throttle.PressureAwareThroughputController(145): 662ddac1568b4f798ff296d3ce34f963#family1#compaction#166 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:07,648 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315cfac801a63146848ca1f9393ce5cc6e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512315cfac801a63146848ca1f9393ce5cc6e 2023-02-09 22:56:07,649 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,682 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:56:07,682 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3215ecc8 to 127.0.0.1:59035 2023-02-09 22:56:07,683 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:07,683 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:56:07,683 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ea8e728 to 127.0.0.1:59035 2023-02-09 22:56:07,683 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:07,697 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/919dbe6d683747599d18ee1f2db89978 first=Optional[r1] last=Optional[r4] 2023-02-09 22:56:07,697 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/919dbe6d683747599d18ee1f2db89978}] 2023-02-09 22:56:07,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/919dbe6d683747599d18ee1f2db89978 for inclusion in 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-09 22:56:07,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:56:07,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:07,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/919dbe6d683747599d18ee1f2db89978 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9bv55jgqo4bdfojf53118jl5j60kj1gschd8blackcjsk0vh8po70akf8msf1cn7/family1/919dbe6d683747599d18ee1f2db89978 2023-02-09 22:56:07,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9bv55jgqo4bdfojf53118jl5j60kj1gschd8blackcjsk0vh8po70akf8msf1cn7/family1/919dbe6d683747599d18ee1f2db89978 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e95e8453b7fd4a7b863bb6bae4ada2c0_SeqId_46_ 2023-02-09 22:56:07,704 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9bv55jgqo4bdfojf53118jl5j60kj1gschd8blackcjsk0vh8po70akf8msf1cn7/family1/919dbe6d683747599d18ee1f2db89978 into 662ddac1568b4f798ff296d3ce34f963/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e95e8453b7fd4a7b863bb6bae4ada2c0_SeqId_46_ - updating store file list. 2023-02-09 22:56:07,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e95e8453b7fd4a7b863bb6bae4ada2c0_SeqId_46_ into 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9bv55jgqo4bdfojf53118jl5j60kj1gschd8blackcjsk0vh8po70akf8msf1cn7/family1/919dbe6d683747599d18ee1f2db89978 into 662ddac1568b4f798ff296d3ce34f963/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e95e8453b7fd4a7b863bb6bae4ada2c0_SeqId_46_) 2023-02-09 22:56:07,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9bv55jgqo4bdfojf53118jl5j60kj1gschd8blackcjsk0vh8po70akf8msf1cn7/family1/919dbe6d683747599d18ee1f2db89978 2023-02-09 22:56:07,712 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,712 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:07,714 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120188f6ee6f9f654755a7c6b1de17ec391a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120188f6ee6f9f654755a7c6b1de17ec391a 2023-02-09 22:56:07,715 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120553f4d0b2c84f4b00b07cc6dca8eeaec7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120553f4d0b2c84f4b00b07cc6dca8eeaec7 2023-02-09 22:56:07,716 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228b16f775bde2647bf8f3206842591a21e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228b16f775bde2647bf8f3206842591a21e 2023-02-09 22:56:07,717 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231174401d29629463ea15228d5abe260fd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231174401d29629463ea15228d5abe260fd 2023-02-09 22:56:07,717 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-09 22:56:07,717 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-09 22:56:07,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,717 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,750 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113092cf342d49734c669cdbd1580fd813d7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113092cf342d49734c669cdbd1580fd813d7 2023-02-09 22:56:07,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,773 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c288ddf5907d44c1bf0acfa4a280d75a first=Optional[r00] last=Optional[r8] 2023-02-09 22:56:07,774 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c288ddf5907d44c1bf0acfa4a280d75a}] 2023-02-09 22:56:07,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c288ddf5907d44c1bf0acfa4a280d75a for inclusion in 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-09 22:56:07,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last= 2023-02-09 22:56:07,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:07,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c288ddf5907d44c1bf0acfa4a280d75a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__iegp84d9mucrngfma1m7972529mmshg6aubrhfuotrrlv9k6qi011ieve8jd78t0/family1/c288ddf5907d44c1bf0acfa4a280d75a 2023-02-09 22:56:07,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__iegp84d9mucrngfma1m7972529mmshg6aubrhfuotrrlv9k6qi011ieve8jd78t0/family1/c288ddf5907d44c1bf0acfa4a280d75a as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e86ec9ab23a648d7ae2e46d2ce072e88_SeqId_48_ 2023-02-09 22:56:07,780 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__iegp84d9mucrngfma1m7972529mmshg6aubrhfuotrrlv9k6qi011ieve8jd78t0/family1/c288ddf5907d44c1bf0acfa4a280d75a into 662ddac1568b4f798ff296d3ce34f963/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e86ec9ab23a648d7ae2e46d2ce072e88_SeqId_48_ - updating store file list. 2023-02-09 22:56:07,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e86ec9ab23a648d7ae2e46d2ce072e88_SeqId_48_ into 662ddac1568b4f798ff296d3ce34f963/family1 2023-02-09 22:56:07,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__iegp84d9mucrngfma1m7972529mmshg6aubrhfuotrrlv9k6qi011ieve8jd78t0/family1/c288ddf5907d44c1bf0acfa4a280d75a into 662ddac1568b4f798ff296d3ce34f963/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/e86ec9ab23a648d7ae2e46d2ce072e88_SeqId_48_) 2023-02-09 22:56:07,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMinorCompactionWithMonthlyPolicy__iegp84d9mucrngfma1m7972529mmshg6aubrhfuotrrlv9k6qi011ieve8jd78t0/family1/c288ddf5907d44c1bf0acfa4a280d75a 2023-02-09 22:56:07,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.SortedCompactionPolicy(75): Selecting compaction from 13 store files, 10 compacting, 2 eligible, 16 blocking 2023-02-09 22:56:07,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-09 22:56:07,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-09 22:56:07,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.CompactSplit(415): Not compacting testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. because compaction request was cancelled 2023-02-09 22:56:07,786 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,787 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:07,788 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205a0dde6ee719419ca3c796f7a42d4d01 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511205a0dde6ee719419ca3c796f7a42d4d01 2023-02-09 22:56:07,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283bde111c768743ff804c93a5513e437b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283bde111c768743ff804c93a5513e437b 2023-02-09 22:56:07,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e9ae63c27b894543abc5e5dde5a4c882 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e9ae63c27b894543abc5e5dde5a4c882 2023-02-09 22:56:07,790 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-09 22:56:07,790 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-09 22:56:07,790 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,790 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,790 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-09 22:56:07,790 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-09 22:56:07,790 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:56:07,793 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-09 22:56:07,794 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.4880 sec 2023-02-09 22:56:07,928 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=259 (was 250) - Thread LEAK? -, OpenFileDescriptor=649 (was 629) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=145 (was 154), ProcessCount=168 (was 168), AvailableMemoryMB=10885 (was 11246) 2023-02-09 22:56:07,941 INFO [Listener at localhost/33819] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=259, OpenFileDescriptor=649, MaxFileDescriptor=60000, SystemLoadAverage=145, ProcessCount=168, AvailableMemoryMB=10884 2023-02-09 22:56:07,970 DEBUG [Listener at localhost/33819] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-09 22:56:08,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.HMaster$4(2102): Client=jenkins//172.31.2.10 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-09 22:56:08,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-09 22:56:08,038 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 22:56:08,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(697): Client=jenkins//172.31.2.10 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-09 22:56:08,039 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 22:56:08,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639 empty. 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8 empty. 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691 empty. 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:08,041 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:08,042 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-09 22:56:08,042 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/.tmp/family1/2a291da5a89748569c5efc6c767003f2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/2a291da5a89748569c5efc6c767003f2 2023-02-09 22:56:08,047 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 662ddac1568b4f798ff296d3ce34f963/family1 of 662ddac1568b4f798ff296d3ce34f963 into 2a291da5a89748569c5efc6c767003f2(size=6.6 K), total size for store is 344.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:56:08,047 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 662ddac1568b4f798ff296d3ce34f963: 2023-02-09 22:56:08,047 INFO [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675983367188; duration=0sec 2023-02-09 22:56:08,047 DEBUG [RS:0;jenkins-hbase9:39093-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 22:56:08,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_] to archive 2023-02-09 22:56:08,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:08,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1adf88689613483fb5db7f2d9ca370ab 2023-02-09 22:56:08,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1839ff985f6d40a69af736764e2a1884 2023-02-09 22:56:08,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/1682e7f63e3347e7bc2da327277fa682 2023-02-09 22:56:08,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c32bc8169f464428b05c595053b1ecfd 2023-02-09 22:56:08,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/b205dd1690564a39bf70d1131e8394cc 2023-02-09 22:56:08,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/dde0eb45693540d19b0e2a68f62ec3e1 2023-02-09 22:56:08,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/68ce072936e44a56ad816d321f405e22 2023-02-09 22:56:08,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/8a3e5002ffa04344a5e2c2ba4593229d 2023-02-09 22:56:08,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/c702758d31ed49fe899688ba1309a232 2023-02-09 22:56:08,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_ to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/family1/35441dff18c04f1cb6836c8ac0943cc3_SeqId_44_ 2023-02-09 22:56:08,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:08,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:08,449 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-09 22:56:08,450 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => fd5b78488926ea4ca842ecc51d27b639, NAME => 'testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:56:08,451 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 30b34a79276f016bf9925a49272eb691, NAME => 'testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:56:08,451 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => 88ca4681719f241d0781e2d53e6d34d8, NAME => 'testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/.tmp 2023-02-09 22:56:08,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:08,856 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:08,856 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:08,856 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing 88ca4681719f241d0781e2d53e6d34d8, disabling compactions & flushes 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 30b34a79276f016bf9925a49272eb691, disabling compactions & flushes 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing fd5b78488926ea4ca842ecc51d27b639, disabling compactions & flushes 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. after waiting 0 ms 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. after waiting 0 ms 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. after waiting 0 ms 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for fd5b78488926ea4ca842ecc51d27b639: Waiting for close lock at 1675983368857Disabling compacts and flushes for region at 1675983368857Disabling writes for close at 1675983368857Writing region close event to WAL at 1675983368857Closed at 1675983368857 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:08,857 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 30b34a79276f016bf9925a49272eb691: Waiting for close lock at 1675983368857Disabling compacts and flushes for region at 1675983368857Disabling writes for close at 1675983368857Writing region close event to WAL at 1675983368857Closed at 1675983368857 2023-02-09 22:56:08,857 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for 88ca4681719f241d0781e2d53e6d34d8: Waiting for close lock at 1675983368857Disabling compacts and flushes for region at 1675983368857Disabling writes for close at 1675983368857Writing region close event to WAL at 1675983368857Closed at 1675983368857 2023-02-09 22:56:08,860 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 22:56:08,860 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983368860"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983368860"}]},"ts":"1675983368860"} 2023-02-09 22:56:08,860 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983368860"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983368860"}]},"ts":"1675983368860"} 2023-02-09 22:56:08,860 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983368860"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675983368860"}]},"ts":"1675983368860"} 2023-02-09 22:56:08,862 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-09 22:56:08,863 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 22:56:08,863 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983368863"}]},"ts":"1675983368863"} 2023-02-09 22:56:08,864 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-09 22:56:08,867 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fd5b78488926ea4ca842ecc51d27b639, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ca4681719f241d0781e2d53e6d34d8, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30b34a79276f016bf9925a49272eb691, ASSIGN}] 2023-02-09 22:56:08,869 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30b34a79276f016bf9925a49272eb691, ASSIGN 2023-02-09 22:56:08,869 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fd5b78488926ea4ca842ecc51d27b639, ASSIGN 2023-02-09 22:56:08,869 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ca4681719f241d0781e2d53e6d34d8, ASSIGN 2023-02-09 22:56:08,870 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30b34a79276f016bf9925a49272eb691, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:56:08,870 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ca4681719f241d0781e2d53e6d34d8, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:56:08,870 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fd5b78488926ea4ca842ecc51d27b639, ASSIGN; state=OFFLINE, location=jenkins-hbase9.apache.org,39093,1675983215026; forceNewPlan=false, retain=false 2023-02-09 22:56:09,022 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=88ca4681719f241d0781e2d53e6d34d8, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,022 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=fd5b78488926ea4ca842ecc51d27b639, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,022 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983369022"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983369022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983369022"}]},"ts":"1675983369022"} 2023-02-09 22:56:09,022 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30b34a79276f016bf9925a49272eb691, regionState=OPENING, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,023 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983369022"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983369022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983369022"}]},"ts":"1675983369022"} 2023-02-09 22:56:09,023 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983369022"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675983369022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675983369022"}]},"ts":"1675983369022"} 2023-02-09 22:56:09,024 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure 88ca4681719f241d0781e2d53e6d34d8, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:56:09,024 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure fd5b78488926ea4ca842ecc51d27b639, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:56:09,025 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 30b34a79276f016bf9925a49272eb691, server=jenkins-hbase9.apache.org,39093,1675983215026}] 2023-02-09 22:56:09,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:09,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 88ca4681719f241d0781e2d53e6d34d8, NAME => 'testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.', STARTKEY => '1', ENDKEY => '2'} 2023-02-09 22:56:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,202 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,204 DEBUG [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1 2023-02-09 22:56:09,204 DEBUG [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1 2023-02-09 22:56:09,204 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88ca4681719f241d0781e2d53e6d34d8 columnFamilyName family1 2023-02-09 22:56:09,205 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] regionserver.HStore(310): Store=88ca4681719f241d0781e2d53e6d34d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,205 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,206 DEBUG [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2 2023-02-09 22:56:09,206 DEBUG [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2 2023-02-09 22:56:09,207 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 88ca4681719f241d0781e2d53e6d34d8 columnFamilyName family2 2023-02-09 22:56:09,208 INFO [StoreOpener-88ca4681719f241d0781e2d53e6d34d8-1] regionserver.HStore(310): Store=88ca4681719f241d0781e2d53e6d34d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:56:09,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:09,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:56:09,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 88ca4681719f241d0781e2d53e6d34d8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75007004, jitterRate=0.11769145727157593}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:56:09,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 88ca4681719f241d0781e2d53e6d34d8: Running coprocessor pre-open hook at 1675983369181Writing region info on filesystem at 1675983369181Initializing all the Stores at 1675983369182 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369182Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369182Cleaning up temporary data from old regions at 1675983369209 (+27 ms)Cleaning up detritus from prior splits at 1675983369209Running coprocessor post-open hooks at 1675983369212 (+3 ms)Region opened successfully at 1675983369212 2023-02-09 22:56:09,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8., pid=120, masterSystemTime=1675983369175 2023-02-09 22:56:09,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:09,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:09,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:09,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fd5b78488926ea4ca842ecc51d27b639, NAME => 'testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.', STARTKEY => '', ENDKEY => '1'} 2023-02-09 22:56:09,215 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=88ca4681719f241d0781e2d53e6d34d8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,215 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675983369215"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983369215"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983369215"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983369215"}]},"ts":"1675983369215"} 2023-02-09 22:56:09,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:09,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-02-09 22:56:09,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure 88ca4681719f241d0781e2d53e6d34d8, server=jenkins-hbase9.apache.org,39093,1675983215026 in 192 msec 2023-02-09 22:56:09,218 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=88ca4681719f241d0781e2d53e6d34d8, ASSIGN in 350 msec 2023-02-09 22:56:09,235 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,236 DEBUG [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1 2023-02-09 22:56:09,237 DEBUG [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1 2023-02-09 22:56:09,237 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd5b78488926ea4ca842ecc51d27b639 columnFamilyName family1 2023-02-09 22:56:09,237 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] regionserver.HStore(310): Store=fd5b78488926ea4ca842ecc51d27b639/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,238 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,239 DEBUG [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2 2023-02-09 22:56:09,239 DEBUG [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2 2023-02-09 22:56:09,239 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fd5b78488926ea4ca842ecc51d27b639 columnFamilyName family2 2023-02-09 22:56:09,240 INFO [StoreOpener-fd5b78488926ea4ca842ecc51d27b639-1] regionserver.HStore(310): Store=fd5b78488926ea4ca842ecc51d27b639/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:56:09,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:09,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:56:09,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened fd5b78488926ea4ca842ecc51d27b639; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59522557, jitterRate=-0.11304478347301483}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:56:09,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for fd5b78488926ea4ca842ecc51d27b639: Running coprocessor pre-open hook at 1675983369215Writing region info on filesystem at 1675983369215Initializing all the Stores at 1675983369215Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369216 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369216Cleaning up temporary data from old regions at 1675983369241 (+25 ms)Cleaning up detritus from prior splits at 1675983369241Running coprocessor post-open hooks at 1675983369244 (+3 ms)Region opened successfully at 1675983369244 2023-02-09 22:56:09,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639., pid=121, masterSystemTime=1675983369175 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:09,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:09,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 30b34a79276f016bf9925a49272eb691, NAME => 'testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.', STARTKEY => '2', ENDKEY => ''} 2023-02-09 22:56:09,246 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=fd5b78488926ea4ca842ecc51d27b639, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7890): checking encryption for 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,246 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983369246"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983369246"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983369246"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983369246"}]},"ts":"1675983369246"} 2023-02-09 22:56:09,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(7893): checking classloading for 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,249 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-02-09 22:56:09,249 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure fd5b78488926ea4ca842ecc51d27b639, server=jenkins-hbase9.apache.org,39093,1675983215026 in 223 msec 2023-02-09 22:56:09,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fd5b78488926ea4ca842ecc51d27b639, ASSIGN in 382 msec 2023-02-09 22:56:09,267 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,268 DEBUG [StoreOpener-30b34a79276f016bf9925a49272eb691-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1 2023-02-09 22:56:09,268 DEBUG [StoreOpener-30b34a79276f016bf9925a49272eb691-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1 2023-02-09 22:56:09,269 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30b34a79276f016bf9925a49272eb691 columnFamilyName family1 2023-02-09 22:56:09,269 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] regionserver.HStore(310): Store=30b34a79276f016bf9925a49272eb691/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,269 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,270 DEBUG [StoreOpener-30b34a79276f016bf9925a49272eb691-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2 2023-02-09 22:56:09,271 DEBUG [StoreOpener-30b34a79276f016bf9925a49272eb691-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2 2023-02-09 22:56:09,271 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30b34a79276f016bf9925a49272eb691 columnFamilyName family2 2023-02-09 22:56:09,271 INFO [StoreOpener-30b34a79276f016bf9925a49272eb691-1] regionserver.HStore(310): Store=30b34a79276f016bf9925a49272eb691/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 22:56:09,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 22:56:09,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1054): writing seq id for 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:09,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 22:56:09,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1071): Opened 30b34a79276f016bf9925a49272eb691; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73783098, jitterRate=0.09945383667945862}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 22:56:09,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(964): Region open journal for 30b34a79276f016bf9925a49272eb691: Running coprocessor pre-open hook at 1675983369246Writing region info on filesystem at 1675983369246Initializing all the Stores at 1675983369247 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369247Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675983369247Cleaning up temporary data from old regions at 1675983369273 (+26 ms)Cleaning up detritus from prior splits at 1675983369273Running coprocessor post-open hooks at 1675983369277 (+4 ms)Region opened successfully at 1675983369277 2023-02-09 22:56:09,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691., pid=122, masterSystemTime=1675983369175 2023-02-09 22:56:09,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:09,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase9:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:09,279 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30b34a79276f016bf9925a49272eb691, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:09,280 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675983369279"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675983369279"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675983369279"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675983369279"}]},"ts":"1675983369279"} 2023-02-09 22:56:09,282 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-02-09 22:56:09,282 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 30b34a79276f016bf9925a49272eb691, server=jenkins-hbase9.apache.org,39093,1675983215026 in 256 msec 2023-02-09 22:56:09,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-02-09 22:56:09,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30b34a79276f016bf9925a49272eb691, ASSIGN in 415 msec 2023-02-09 22:56:09,284 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 22:56:09,284 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675983369284"}]},"ts":"1675983369284"} 2023-02-09 22:56:09,285 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-09 22:56:09,287 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 22:56:09,288 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2510 sec 2023-02-09 22:56:10,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-09 22:56:10,144 INFO [Listener at localhost/33819] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-09 22:56:10,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:10,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:10,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:10,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:10,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 22:56:10,155 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 22:56:10,155 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:10,155 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 22:56:10,155 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 22:56:10,155 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:10,155 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:10,164 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:10,164 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:10,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:10,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:10,165 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:10,165 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:10,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:10,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:10,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:10,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 22:56:10,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 22:56:10,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 22:56:10,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:10,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 22:56:10,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:10,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. started... 2023-02-09 22:56:10,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:10,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:56:10,167 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:10,166 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. started... 2023-02-09 22:56:10,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:10,167 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 88ca4681719f241d0781e2d53e6d34d8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:56:10,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. started... 2023-02-09 22:56:10,168 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 30b34a79276f016bf9925a49272eb691 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:56:10,168 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing fd5b78488926ea4ca842ecc51d27b639 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-09 22:56:10,616 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302093f6b9e377c8e46a38426c9b908e9170c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f6b9e377c8e46a38426c9b908e9170c 2023-02-09 22:56:10,620 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302090898b11ed15a4fe59b6b93a2d2f30ce0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302090898b11ed15a4fe59b6b93a2d2f30ce0 2023-02-09 22:56:10,630 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302094a6192520d764d44b01f212c453c93dd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302094a6192520d764d44b01f212c453c93dd 2023-02-09 22:56:10,646 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/19bbc4f389bc48cb8d597a605e97f780 2023-02-09 22:56:11,041 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/365d36e59ff843999ee91e17fb335c55 2023-02-09 22:56:11,059 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/b6eb68f74db142dfafe299f327046100 2023-02-09 22:56:11,074 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209ae05c648567540f3ad6ef6f8f4b62d08 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209ae05c648567540f3ad6ef6f8f4b62d08 2023-02-09 22:56:11,081 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209e570d1ae3eb24042b42d7557eba982c4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209e570d1ae3eb24042b42d7557eba982c4 2023-02-09 22:56:11,479 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/b019de43caab438b9af73973f1609bb6 2023-02-09 22:56:11,482 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/365d36e59ff843999ee91e17fb335c55 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55 2023-02-09 22:56:11,486 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/5d2aa28be6ea41dcad82a682b97e011b 2023-02-09 22:56:11,489 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/19bbc4f389bc48cb8d597a605e97f780 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780 2023-02-09 22:56:11,493 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209925c9a12c8ef45cbae8c84680349b93f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230209925c9a12c8ef45cbae8c84680349b93f 2023-02-09 22:56:11,498 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/6965c0d6e7154624883e74e626bcef73 2023-02-09 22:56:11,503 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/b6eb68f74db142dfafe299f327046100 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100 2023-02-09 22:56:11,511 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 22:56:11,512 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/b019de43caab438b9af73973f1609bb6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6 2023-02-09 22:56:11,519 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:56:11,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 30b34a79276f016bf9925a49272eb691 in 1352ms, sequenceid=5, compaction requested=false 2023-02-09 22:56:11,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-09 22:56:11,521 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 22:56:11,521 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/5d2aa28be6ea41dcad82a682b97e011b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b 2023-02-09 22:56:11,522 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:11,522 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:11,522 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:56:11,526 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:56:11,526 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 88ca4681719f241d0781e2d53e6d34d8 in 1359ms, sequenceid=5, compaction requested=false 2023-02-09 22:56:11,527 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:11,527 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:11,527 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:56:11,545 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100, entries=4, sequenceid=5, filesize=5.5 K 2023-02-09 22:56:11,546 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/6965c0d6e7154624883e74e626bcef73 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73 2023-02-09 22:56:11,550 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73, entries=2, sequenceid=5, filesize=5.1 K 2023-02-09 22:56:11,551 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fd5b78488926ea4ca842ecc51d27b639 in 1383ms, sequenceid=5, compaction requested=false 2023-02-09 22:56:11,551 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:11,551 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:11,551 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:56:11,551 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:56:11,552 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,552 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 22:56:11,552 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 22:56:11,554 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,554 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:11,554 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,554 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:11,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:11,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:11,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 22:56:11,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@290fd3f1[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:11,556 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 22:56:11,556 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,557 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,557 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 22:56:11,557 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 22:56:11,557 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:11,557 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,557 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:11,559 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,559 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 22:56:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:11,559 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:11,559 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 22:56:11,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:11,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:11,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:11,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:11,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 22:56:11,562 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 22:56:11,562 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:11,562 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:11,562 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 22:56:11,562 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:11,565 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,565 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:11,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,565 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:11,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:11,565 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,566 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:11,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:11,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,569 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:11,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:11,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 22:56:11,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:11,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:11,569 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:11,570 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:11,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:11,570 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:11,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:11,570 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,570 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:11,670 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:11,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:11,672 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:11,673 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:11,673 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:11,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:11,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:11,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 22:56:11,676 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 22:56:11,676 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:11,676 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 22:56:11,676 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 22:56:11,677 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,677 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:11,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,678 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:11,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:11,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:11,678 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:11,679 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:11,679 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 22:56:11,679 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 22:56:11,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 22:56:11,679 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 22:56:11,679 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:11,679 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:11,680 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:56:11,680 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. started... 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. started... 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:11,680 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. started... 2023-02-09 22:56:11,680 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 30b34a79276f016bf9925a49272eb691 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:56:11,680 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 88ca4681719f241d0781e2d53e6d34d8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:56:11,680 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing fd5b78488926ea4ca842ecc51d27b639 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-09 22:56:12,130 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209cd85cdc3ba4844efad5c7b686653b868 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209cd85cdc3ba4844efad5c7b686653b868 2023-02-09 22:56:12,133 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209ae4375545e534f8dbf5e18d3f7ded36a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae4375545e534f8dbf5e18d3f7ded36a 2023-02-09 22:56:12,133 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209bdba32ca66b347d9803a26d11aa2aa94 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209bdba32ca66b347d9803a26d11aa2aa94 2023-02-09 22:56:12,147 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/e1ce276dcede4818b14132986aadc816 2023-02-09 22:56:12,153 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/1688be162c4a483c97bef1b5403192d5 2023-02-09 22:56:12,545 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/b92dd1715f7143d6b2cf2be1e9a253c8 2023-02-09 22:56:12,577 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209650cd15701344c9f94c2ad57887dcafe to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230209650cd15701344c9f94c2ad57887dcafe 2023-02-09 22:56:12,592 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209943e1b689a1d47d69b445962f8c53d04 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209943e1b689a1d47d69b445962f8c53d04 2023-02-09 22:56:12,977 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302094817436cce8249e2b6e4385c9adb1be1 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302094817436cce8249e2b6e4385c9adb1be1 2023-02-09 22:56:12,981 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/502826c135f6479c80447de91288a5d5 2023-02-09 22:56:12,981 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/2d39913104e440279f4a547c6da68ecf 2023-02-09 22:56:12,985 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/b92dd1715f7143d6b2cf2be1e9a253c8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8 2023-02-09 22:56:12,985 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/e1ce276dcede4818b14132986aadc816 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816 2023-02-09 22:56:13,004 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/ba50b38770f74ab88e3fe1c019101612 2023-02-09 22:56:13,008 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 22:56:13,009 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/1688be162c4a483c97bef1b5403192d5 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5 2023-02-09 22:56:13,009 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 22:56:13,009 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/502826c135f6479c80447de91288a5d5 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5 2023-02-09 22:56:13,010 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/2d39913104e440279f4a547c6da68ecf as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf 2023-02-09 22:56:13,013 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:56:13,013 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:56:13,014 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fd5b78488926ea4ca842ecc51d27b639 in 1334ms, sequenceid=9, compaction requested=false 2023-02-09 22:56:13,014 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:13,014 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 30b34a79276f016bf9925a49272eb691 in 1334ms, sequenceid=9, compaction requested=false 2023-02-09 22:56:13,014 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:13,014 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:13,014 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:56:13,014 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:13,014 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:56:13,031 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5, entries=4, sequenceid=9, filesize=5.5 K 2023-02-09 22:56:13,032 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/ba50b38770f74ab88e3fe1c019101612 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612 2023-02-09 22:56:13,035 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612, entries=2, sequenceid=9, filesize=5.1 K 2023-02-09 22:56:13,036 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 88ca4681719f241d0781e2d53e6d34d8 in 1356ms, sequenceid=9, compaction requested=false 2023-02-09 22:56:13,036 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:13,036 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:13,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:56:13,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:56:13,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 22:56:13,036 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 22:56:13,038 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,038 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:13,038 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,038 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:13,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 22:56:13,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41de2ec0[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:13,039 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 22:56:13,039 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,042 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,042 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 22:56:13,042 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 22:56:13,042 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:13,042 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,042 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:13,043 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,044 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 22:56:13,044 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:13,044 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:13,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:13,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:13,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 22:56:13,046 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 22:56:13,046 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:13,046 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:13,046 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 22:56:13,046 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:13,048 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,048 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:13,048 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:13,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:13,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:13,048 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:13,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,054 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:13,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:13,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:13,056 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:13,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 22:56:13,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:13,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:13,057 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:13,056 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,057 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:13,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:13,057 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,057 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:13,057 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,157 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:13,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:13,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:13,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:13,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:13,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:13,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:13,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 22:56:13,162 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 22:56:13,162 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:13,162 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 22:56:13,162 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 22:56:13,163 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,163 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:13,165 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,165 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:13,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:13,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:13,165 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:13,165 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:13,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:13,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:13,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:13,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:13,166 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 22:56:13,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 22:56:13,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 22:56:13,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 22:56:13,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:13,166 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:13,167 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:56:13,167 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. started... 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. started... 2023-02-09 22:56:13,167 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 88ca4681719f241d0781e2d53e6d34d8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:56:13,167 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 30b34a79276f016bf9925a49272eb691 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:56:13,167 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. started... 2023-02-09 22:56:13,167 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing fd5b78488926ea4ca842ecc51d27b639 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-09 22:56:13,215 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020968658d16e1094af29edf2426bf8206c9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020968658d16e1094af29edf2426bf8206c9 2023-02-09 22:56:13,608 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209cc60415d021a46d7878361f787c79e55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cc60415d021a46d7878361f787c79e55 2023-02-09 22:56:13,622 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/9b57c2358ec749028d49600613792851 2023-02-09 22:56:13,626 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020930517027752c4cbaa08857a5761d5211 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020930517027752c4cbaa08857a5761d5211 2023-02-09 22:56:13,630 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/96de981dc45b48d3a531e3e232698ab3 2023-02-09 22:56:14,050 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/aaee4ef1e80f4ad6981076c449b86895 2023-02-09 22:56:14,065 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209540ef0cbaebf4820a92fde4fe757af4d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209540ef0cbaebf4820a92fde4fe757af4d 2023-02-09 22:56:14,067 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209940971dd01d94969a998333c42a2f4a6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230209940971dd01d94969a998333c42a2f4a6 2023-02-09 22:56:14,481 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/079dcfbadb644b62a1e533eced2b7903 2023-02-09 22:56:14,481 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/91bc3aff97de446c869e253b9b8059ec 2023-02-09 22:56:14,485 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/96de981dc45b48d3a531e3e232698ab3 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3 2023-02-09 22:56:14,485 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/9b57c2358ec749028d49600613792851 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851 2023-02-09 22:56:14,488 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302097d0b147507a34dec83894664fb064fbb to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302097d0b147507a34dec83894664fb064fbb 2023-02-09 22:56:14,494 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/f018d79cb67f406d82ca8fb7f60f43f8 2023-02-09 22:56:14,498 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/aaee4ef1e80f4ad6981076c449b86895 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895 2023-02-09 22:56:14,509 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 22:56:14,510 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/91bc3aff97de446c869e253b9b8059ec as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec 2023-02-09 22:56:14,513 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:56:14,514 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 88ca4681719f241d0781e2d53e6d34d8 in 1347ms, sequenceid=13, compaction requested=true 2023-02-09 22:56:14,514 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:14,514 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:14,514 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:56:14,518 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 22:56:14,518 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/079dcfbadb644b62a1e533eced2b7903 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903 2023-02-09 22:56:14,520 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895, entries=4, sequenceid=13, filesize=5.5 K 2023-02-09 22:56:14,520 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/f018d79cb67f406d82ca8fb7f60f43f8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8 2023-02-09 22:56:14,522 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:56:14,522 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fd5b78488926ea4ca842ecc51d27b639 in 1355ms, sequenceid=13, compaction requested=true 2023-02-09 22:56:14,522 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:14,522 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:14,523 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:56:14,524 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8, entries=2, sequenceid=13, filesize=5.1 K 2023-02-09 22:56:14,524 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 30b34a79276f016bf9925a49272eb691 in 1357ms, sequenceid=13, compaction requested=true 2023-02-09 22:56:14,524 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:14,524 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:14,524 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:56:14,525 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:56:14,525 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,525 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 22:56:14,525 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 22:56:14,526 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,526 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:14,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:14,526 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,526 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:14,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 22:56:14,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8c34c39[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:14,528 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 22:56:14,528 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,529 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,529 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 22:56:14,529 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 22:56:14,529 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:14,529 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,529 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:14,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 22:56:14,531 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,531 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:14,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:14,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 22:56:14,533 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 22:56:14,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:14,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:14,533 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 22:56:14,533 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:14,535 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,535 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,536 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:14,536 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,537 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:14,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:14,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:14,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 22:56:14,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:14,539 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:14,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 22:56:14,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:14,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:14,539 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:14,540 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:14,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:14,540 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,540 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:14,540 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,640 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:14,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:14,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:14,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:14,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. with WAL disabled. Data may be lost in the event of a crash. 2023-02-09 22:56:14,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:14,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:14,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 22:56:14,646 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 22:56:14,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:14,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 22:56:14,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 22:56:14,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,646 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:14,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,654 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:14,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:14,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:14,654 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:14,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:14,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:14,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:14,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:14,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 22:56:14,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 22:56:14,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 22:56:14,655 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 22:56:14,655 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:14,655 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 22:56:14,655 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:14,656 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. started... 2023-02-09 22:56:14,656 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:14,656 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:14,656 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:56:14,656 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:14,656 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 88ca4681719f241d0781e2d53e6d34d8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:56:14,656 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. started... 2023-02-09 22:56:14,656 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. started... 2023-02-09 22:56:14,656 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing fd5b78488926ea4ca842ecc51d27b639 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:56:14,656 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 30b34a79276f016bf9925a49272eb691 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-09 22:56:14,705 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302095a4005c8f72c4cf8b75ed5a15fad80f2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095a4005c8f72c4cf8b75ed5a15fad80f2 2023-02-09 22:56:14,720 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 22:56:15,105 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302099446113eefd843088a567eac8d6d10d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302099446113eefd843088a567eac8d6d10d2 2023-02-09 22:56:15,106 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209cdd8462f9e3a4941acc448b71b347388 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cdd8462f9e3a4941acc448b71b347388 2023-02-09 22:56:15,120 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/edbc9d939fec44fa8ee7549204250481 2023-02-09 22:56:15,120 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/271f15b2416944dcb559248f23710805 2023-02-09 22:56:15,519 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/23ed05d924c9430eaaf4ad39e87bbedd 2023-02-09 22:56:15,554 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302092b2a781db3c44ee0a64fe5a896740f49 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302092b2a781db3c44ee0a64fe5a896740f49 2023-02-09 22:56:15,568 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209da4a53f89b3a49198cddc312e63ce4d4 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230209da4a53f89b3a49198cddc312e63ce4d4 2023-02-09 22:56:15,952 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020987f6212f07674ab68d82fce6635aca7f to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023020987f6212f07674ab68d82fce6635aca7f 2023-02-09 22:56:15,956 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/f357c0d7ea314c2aa1b2eb71deb82921 2023-02-09 22:56:15,960 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/23ed05d924c9430eaaf4ad39e87bbedd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd 2023-02-09 22:56:15,966 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/48579322234e43599dfb081d8595a225 2023-02-09 22:56:15,970 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/edbc9d939fec44fa8ee7549204250481 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481 2023-02-09 22:56:15,973 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/70abbb920ba5490388cf9ce292a73563 2023-02-09 22:56:15,978 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/271f15b2416944dcb559248f23710805 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805 2023-02-09 22:56:15,996 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 22:56:15,997 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/f357c0d7ea314c2aa1b2eb71deb82921 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921 2023-02-09 22:56:15,999 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 22:56:16,000 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/48579322234e43599dfb081d8595a225 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225 2023-02-09 22:56:16,001 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:56:16,002 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fd5b78488926ea4ca842ecc51d27b639 in 1346ms, sequenceid=17, compaction requested=true 2023-02-09 22:56:16,002 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:16,002 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:16,002 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:56:16,003 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:56:16,004 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 30b34a79276f016bf9925a49272eb691 in 1348ms, sequenceid=17, compaction requested=true 2023-02-09 22:56:16,004 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:16,004 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:16,004 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:56:16,009 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805, entries=4, sequenceid=17, filesize=5.5 K 2023-02-09 22:56:16,009 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/70abbb920ba5490388cf9ce292a73563 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563 2023-02-09 22:56:16,012 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563, entries=2, sequenceid=17, filesize=5.1 K 2023-02-09 22:56:16,013 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 88ca4681719f241d0781e2d53e6d34d8 in 1357ms, sequenceid=17, compaction requested=true 2023-02-09 22:56:16,013 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:16,013 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:16,013 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:56:16,013 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:56:16,013 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,013 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 22:56:16,013 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 22:56:16,016 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,016 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:16,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:16,017 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,017 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:16,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 22:56:16,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a11f0d3[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:16,018 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 22:56:16,018 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,019 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,019 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 22:56:16,019 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,019 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:16,019 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 22:56:16,019 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:16,021 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 22:56:16,021 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,021 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:16,021 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 22:56:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:16,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 22:56:16,023 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 22:56:16,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:16,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:16,023 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 22:56:16,023 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:16,025 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,025 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,025 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:16,025 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:16,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:16,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:16,026 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:16,029 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 22:56:16,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:16,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:16,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:16,029 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,129 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:16,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:16,137 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:16,143 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:16,149 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:16,150 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:16,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(933): Client=jenkins//172.31.2.10 procedure request for: flush-table-proc 2023-02-09 22:56:16,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-09 22:56:16,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-09 22:56:16,232 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-09 22:56:16,232 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:16,232 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-09 22:56:16,232 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-09 22:56:16,232 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,232 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:16,234 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,234 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:16,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:16,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:16,234 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:16,234 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 22:56:16,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:16,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:16,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 22:56:16,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:16,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-09 22:56:16,235 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-09 22:56:16,235 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-09 22:56:16,235 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-09 22:56:16,235 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 22:56:16,235 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-09 22:56:16,235 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:16,235 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. started... 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:16,235 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing fd5b78488926ea4ca842ecc51d27b639 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. started... 2023-02-09 22:56:16,235 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. started... 2023-02-09 22:56:16,236 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 88ca4681719f241d0781e2d53e6d34d8 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:56:16,236 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 30b34a79276f016bf9925a49272eb691 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-09 22:56:16,683 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/5c19b02871e840a6814916d8662a00d6 2023-02-09 22:56:16,684 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/09b2690ca84940a09ef3872844c3e301 2023-02-09 22:56:16,706 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09b2690ca84940a09ef3872844c3e301 2023-02-09 22:56:16,717 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c19b02871e840a6814916d8662a00d6 2023-02-09 22:56:16,734 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/004c1a26eec747989a43b282805991c7 2023-02-09 22:56:16,737 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 004c1a26eec747989a43b282805991c7 2023-02-09 22:56:16,737 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/5c19b02871e840a6814916d8662a00d6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6 2023-02-09 22:56:16,760 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c19b02871e840a6814916d8662a00d6 2023-02-09 22:56:16,760 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 22:56:16,760 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/004c1a26eec747989a43b282805991c7 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7 2023-02-09 22:56:16,763 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 004c1a26eec747989a43b282805991c7 2023-02-09 22:56:16,763 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:56:16,764 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fd5b78488926ea4ca842ecc51d27b639 in 529ms, sequenceid=23, compaction requested=true 2023-02-09 22:56:16,764 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:16,764 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:16,764 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-09 22:56:17,083 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/1fcaef5df9ae4cf1b7c9e358131826a5 2023-02-09 22:56:17,106 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1fcaef5df9ae4cf1b7c9e358131826a5 2023-02-09 22:56:17,128 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/e15cc05720f94587ad04e3e8cb6eee01 2023-02-09 22:56:17,131 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e15cc05720f94587ad04e3e8cb6eee01 2023-02-09 22:56:17,132 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/09b2690ca84940a09ef3872844c3e301 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301 2023-02-09 22:56:17,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09b2690ca84940a09ef3872844c3e301 2023-02-09 22:56:17,154 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 22:56:17,155 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/e15cc05720f94587ad04e3e8cb6eee01 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01 2023-02-09 22:56:17,158 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e15cc05720f94587ad04e3e8cb6eee01 2023-02-09 22:56:17,158 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:56:17,159 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 30b34a79276f016bf9925a49272eb691 in 923ms, sequenceid=23, compaction requested=true 2023-02-09 22:56:17,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:17,159 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:17,159 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-09 22:56:17,918 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/4844adcad596451196c64a029be97bf9 2023-02-09 22:56:17,921 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4844adcad596451196c64a029be97bf9 2023-02-09 22:56:17,921 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/1fcaef5df9ae4cf1b7c9e358131826a5 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5 2023-02-09 22:56:17,944 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1fcaef5df9ae4cf1b7c9e358131826a5 2023-02-09 22:56:17,944 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5, entries=3, sequenceid=23, filesize=5.2 K 2023-02-09 22:56:17,945 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/4844adcad596451196c64a029be97bf9 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9 2023-02-09 22:56:17,948 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4844adcad596451196c64a029be97bf9 2023-02-09 22:56:17,948 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9, entries=1, sequenceid=23, filesize=5.0 K 2023-02-09 22:56:17,949 INFO [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 88ca4681719f241d0781e2d53e6d34d8 in 1713ms, sequenceid=23, compaction requested=true 2023-02-09 22:56:17,949 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:17,949 DEBUG [rs(jenkins-hbase9.apache.org,39093,1675983215026)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:17,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-09 22:56:17,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-09 22:56:17,949 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-09 22:56:17,950 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-09 22:56:17,952 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:17,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:17,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,952 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 22:56:17,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:17,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:17,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:17,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase9.apache.org,39093,1675983215026' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-09 22:56:17,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d562c3b[Count = 0] remaining members to acquire global barrier 2023-02-09 22:56:17,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-09 22:56:17,954 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,957 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,957 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-09 22:56:17,957 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-09 22:56:17,957 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase9.apache.org,39093,1675983215026' in zk 2023-02-09 22:56:17,957 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,957 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 22:56:17,958 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-09 22:56:17,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:17,959 DEBUG [member: 'jenkins-hbase9.apache.org,39093,1675983215026' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-09 22:56:17,963 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:17,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:17,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:17,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase9.apache.org,39093,1675983215026': 2023-02-09 22:56:17,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase9.apache.org,39093,1675983215026' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-09 22:56:17,966 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-09 22:56:17,966 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 22:56:17,966 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 22:56:17,966 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-09 22:56:17,966 INFO [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 22:56:17,969 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,969 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 22:56:17,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 22:56:17,969 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:17,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 22:56:17,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:17,970 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:17,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 22:56:17,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,971 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 22:56:17,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-09 22:56:17,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,973 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,973 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 22:56:17,973 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:17,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 22:56:17,974 DEBUG [(jenkins-hbase9.apache.org,35735,1675983213878)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 22:56:17,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-09 22:56:17,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 22:56:17,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-09 22:56:17,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 22:56:17,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-09 22:56:17,974 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 22:56:18,074 DEBUG [Listener at localhost/33819] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 22:56:18,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35735] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 22:56:18,076 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:18,076 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 88ca4681719f241d0781e2d53e6d34d8/family1 is initiating major compaction (all files) 2023-02-09 22:56:18,076 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:18,076 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:18,076 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 88ca4681719f241d0781e2d53e6d34d8/family1 in testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:18,076 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp, totalSize=27.1 K 2023-02-09 22:56:18,077 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 19bbc4f389bc48cb8d597a605e97f780, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:18,077 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 1688be162c4a483c97bef1b5403192d5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371673 2023-02-09 22:56:18,078 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 9b57c2358ec749028d49600613792851, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373159 2023-02-09 22:56:18,078 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 271f15b2416944dcb559248f23710805, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:18,078 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 1fcaef5df9ae4cf1b7c9e358131826a5, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:18,086 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 88ca4681719f241d0781e2d53e6d34d8#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:18,524 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del 2023-02-09 22:56:18,524 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del 2023-02-09 22:56:19,376 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ff1244e30d14e248426c14e5b47b76b 2023-02-09 22:56:19,377 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family1/9ff1244e30d14e248426c14e5b47b76b as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9ff1244e30d14e248426c14e5b47b76b 2023-02-09 22:56:19,404 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ff1244e30d14e248426c14e5b47b76b 2023-02-09 22:56:19,405 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 88ca4681719f241d0781e2d53e6d34d8/family1 of 88ca4681719f241d0781e2d53e6d34d8 into 9ff1244e30d14e248426c14e5b47b76b(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:56:19,405 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:19,405 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:19,405 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 88ca4681719f241d0781e2d53e6d34d8/family2 is initiating major compaction (all files) 2023-02-09 22:56:19,405 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:19,406 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:19,406 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 88ca4681719f241d0781e2d53e6d34d8/family2 in testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:19,406 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp, totalSize=25.4 K 2023-02-09 22:56:19,406 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 5d2aa28be6ea41dcad82a682b97e011b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:19,407 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting ba50b38770f74ab88e3fe1c019101612, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371673 2023-02-09 22:56:19,407 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 91bc3aff97de446c869e253b9b8059ec, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373159 2023-02-09 22:56:19,407 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 70abbb920ba5490388cf9ce292a73563, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:19,407 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 4844adcad596451196c64a029be97bf9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:19,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5] to archive 2023-02-09 22:56:19,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:19,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/19bbc4f389bc48cb8d597a605e97f780 2023-02-09 22:56:19,415 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 88ca4681719f241d0781e2d53e6d34d8#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:19,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1688be162c4a483c97bef1b5403192d5 2023-02-09 22:56:19,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/9b57c2358ec749028d49600613792851 2023-02-09 22:56:19,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/271f15b2416944dcb559248f23710805 2023-02-09 22:56:19,419 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/1fcaef5df9ae4cf1b7c9e358131826a5 2023-02-09 22:56:19,823 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302093ecde08b36564208b8f14d67236b6a91_del 2023-02-09 22:56:19,823 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302093ecde08b36564208b8f14d67236b6a91_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302093ecde08b36564208b8f14d67236b6a91_del 2023-02-09 22:56:20,635 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86ec508405fc4e8aab48acce442ae737 2023-02-09 22:56:20,636 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/.tmp/family2/86ec508405fc4e8aab48acce442ae737 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/86ec508405fc4e8aab48acce442ae737 2023-02-09 22:56:20,639 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86ec508405fc4e8aab48acce442ae737 2023-02-09 22:56:20,640 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 88ca4681719f241d0781e2d53e6d34d8/family2 of 88ca4681719f241d0781e2d53e6d34d8 into 86ec508405fc4e8aab48acce442ae737(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-09 22:56:20,640 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:20,640 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:20,640 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 30b34a79276f016bf9925a49272eb691/family1 is initiating major compaction (all files) 2023-02-09 22:56:20,641 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:20,641 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:20,641 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 30b34a79276f016bf9925a49272eb691/family1 in testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:20,641 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp, totalSize=27.1 K 2023-02-09 22:56:20,641 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 365d36e59ff843999ee91e17fb335c55, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:20,642 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting b92dd1715f7143d6b2cf2be1e9a253c8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371673 2023-02-09 22:56:20,642 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting aaee4ef1e80f4ad6981076c449b86895, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373159 2023-02-09 22:56:20,642 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting edbc9d939fec44fa8ee7549204250481, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:20,642 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 09b2690ca84940a09ef3872844c3e301, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:20,649 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 30b34a79276f016bf9925a49272eb691#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:20,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9] to archive 2023-02-09 22:56:20,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:20,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/5d2aa28be6ea41dcad82a682b97e011b 2023-02-09 22:56:20,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/ba50b38770f74ab88e3fe1c019101612 2023-02-09 22:56:20,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/91bc3aff97de446c869e253b9b8059ec 2023-02-09 22:56:20,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/70abbb920ba5490388cf9ce292a73563 2023-02-09 22:56:20,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family2/4844adcad596451196c64a029be97bf9 2023-02-09 22:56:21,087 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del 2023-02-09 22:56:21,088 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del 2023-02-09 22:56:21,551 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da3679666689425a969c365af84e10f6 2023-02-09 22:56:21,551 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family1/da3679666689425a969c365af84e10f6 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/da3679666689425a969c365af84e10f6 2023-02-09 22:56:21,573 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da3679666689425a969c365af84e10f6 2023-02-09 22:56:21,574 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30b34a79276f016bf9925a49272eb691/family1 of 30b34a79276f016bf9925a49272eb691 into da3679666689425a969c365af84e10f6(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:56:21,574 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:21,574 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:21,574 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): 30b34a79276f016bf9925a49272eb691/family2 is initiating major compaction (all files) 2023-02-09 22:56:21,575 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:21,575 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:21,575 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of 30b34a79276f016bf9925a49272eb691/family2 in testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:21,575 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp, totalSize=25.4 K 2023-02-09 22:56:21,575 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting b019de43caab438b9af73973f1609bb6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:21,576 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 2d39913104e440279f4a547c6da68ecf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371673 2023-02-09 22:56:21,576 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting f018d79cb67f406d82ca8fb7f60f43f8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373159 2023-02-09 22:56:21,576 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 48579322234e43599dfb081d8595a225, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:21,577 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting e15cc05720f94587ad04e3e8cb6eee01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:21,584 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): 30b34a79276f016bf9925a49272eb691#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:21,590 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209c4c67b93db43465ebb9f24c20c79d3f2_del 2023-02-09 22:56:21,590 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230209c4c67b93db43465ebb9f24c20c79d3f2_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230209c4c67b93db43465ebb9f24c20c79d3f2_del 2023-02-09 22:56:21,601 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd47bba03fb54c689182cec37933c1d2 2023-02-09 22:56:21,601 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/.tmp/family2/fd47bba03fb54c689182cec37933c1d2 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/fd47bba03fb54c689182cec37933c1d2 2023-02-09 22:56:21,605 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fd47bba03fb54c689182cec37933c1d2 2023-02-09 22:56:21,606 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30b34a79276f016bf9925a49272eb691/family2 of 30b34a79276f016bf9925a49272eb691 into fd47bba03fb54c689182cec37933c1d2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:56:21,606 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:21,606 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:21,606 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): fd5b78488926ea4ca842ecc51d27b639/family1 is initiating major compaction (all files) 2023-02-09 22:56:21,606 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:21,606 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:21,607 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of fd5b78488926ea4ca842ecc51d27b639/family1 in testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:21,607 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp, totalSize=27.1 K 2023-02-09 22:56:21,608 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting b6eb68f74db142dfafe299f327046100, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:21,608 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting e1ce276dcede4818b14132986aadc816, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371672 2023-02-09 22:56:21,608 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 96de981dc45b48d3a531e3e232698ab3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373158 2023-02-09 22:56:21,609 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 23ed05d924c9430eaaf4ad39e87bbedd, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:21,609 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 5c19b02871e840a6814916d8662a00d6, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:21,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301] to archive 2023-02-09 22:56:21,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:21,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/365d36e59ff843999ee91e17fb335c55 2023-02-09 22:56:21,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/b92dd1715f7143d6b2cf2be1e9a253c8 2023-02-09 22:56:21,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/aaee4ef1e80f4ad6981076c449b86895 2023-02-09 22:56:21,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/edbc9d939fec44fa8ee7549204250481 2023-02-09 22:56:21,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/09b2690ca84940a09ef3872844c3e301 2023-02-09 22:56:21,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01] to archive 2023-02-09 22:56:21,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:21,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/b019de43caab438b9af73973f1609bb6 2023-02-09 22:56:21,623 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): fd5b78488926ea4ca842ecc51d27b639#family1#compaction#201 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:21,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/2d39913104e440279f4a547c6da68ecf 2023-02-09 22:56:21,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/f018d79cb67f406d82ca8fb7f60f43f8 2023-02-09 22:56:21,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/48579322234e43599dfb081d8595a225 2023-02-09 22:56:21,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family2/e15cc05720f94587ad04e3e8cb6eee01 2023-02-09 22:56:21,668 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del 2023-02-09 22:56:21,668 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del 2023-02-09 22:56:22,520 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b35dcd1bdba435cb5435a3baf9cf3dd 2023-02-09 22:56:22,521 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family1/3b35dcd1bdba435cb5435a3baf9cf3dd as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/3b35dcd1bdba435cb5435a3baf9cf3dd 2023-02-09 22:56:22,544 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b35dcd1bdba435cb5435a3baf9cf3dd 2023-02-09 22:56:22,545 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fd5b78488926ea4ca842ecc51d27b639/family1 of fd5b78488926ea4ca842ecc51d27b639 into 3b35dcd1bdba435cb5435a3baf9cf3dd(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:56:22,545 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:22,545 DEBUG [Listener at localhost/33819] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-09 22:56:22,545 DEBUG [Listener at localhost/33819] regionserver.HStore(1912): fd5b78488926ea4ca842ecc51d27b639/family2 is initiating major compaction (all files) 2023-02-09 22:56:22,545 INFO [Listener at localhost/33819] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 22:56:22,545 INFO [Listener at localhost/33819] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 22:56:22,546 INFO [Listener at localhost/33819] regionserver.HRegion(2258): Starting compaction of fd5b78488926ea4ca842ecc51d27b639/family2 in testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:22,546 INFO [Listener at localhost/33819] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7] into tmpdir=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp, totalSize=25.4 K 2023-02-09 22:56:22,546 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 6965c0d6e7154624883e74e626bcef73, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675983370150 2023-02-09 22:56:22,546 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 502826c135f6479c80447de91288a5d5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675983371672 2023-02-09 22:56:22,547 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 079dcfbadb644b62a1e533eced2b7903, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675983373158 2023-02-09 22:56:22,547 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting f357c0d7ea314c2aa1b2eb71deb82921, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675983374642 2023-02-09 22:56:22,547 DEBUG [Listener at localhost/33819] compactions.Compactor(207): Compacting 004c1a26eec747989a43b282805991c7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-09 22:56:22,555 INFO [Listener at localhost/33819] throttle.PressureAwareThroughputController(145): fd5b78488926ea4ca842ecc51d27b639#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 22:56:22,561 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302098fa770bf600543f5b918a9125f680420_del 2023-02-09 22:56:22,561 INFO [Listener at localhost/33819] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302098fa770bf600543f5b918a9125f680420_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302098fa770bf600543f5b918a9125f680420_del 2023-02-09 22:56:22,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6] to archive 2023-02-09 22:56:22,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:22,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/b6eb68f74db142dfafe299f327046100 2023-02-09 22:56:22,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/e1ce276dcede4818b14132986aadc816 2023-02-09 22:56:22,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/96de981dc45b48d3a531e3e232698ab3 2023-02-09 22:56:22,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/23ed05d924c9430eaaf4ad39e87bbedd 2023-02-09 22:56:22,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/5c19b02871e840a6814916d8662a00d6 2023-02-09 22:56:22,973 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f475e322dc24f479985087fbe87face 2023-02-09 22:56:22,973 DEBUG [Listener at localhost/33819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/.tmp/family2/5f475e322dc24f479985087fbe87face as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/5f475e322dc24f479985087fbe87face 2023-02-09 22:56:22,976 INFO [Listener at localhost/33819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f475e322dc24f479985087fbe87face 2023-02-09 22:56:22,977 INFO [Listener at localhost/33819] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fd5b78488926ea4ca842ecc51d27b639/family2 of fd5b78488926ea4ca842ecc51d27b639 into 5f475e322dc24f479985087fbe87face(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 22:56:22,977 DEBUG [Listener at localhost/33819] regionserver.HRegion(2288): Compaction status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:22,981 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:22,983 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:22,986 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:22,987 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:22,991 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35735] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x11If\x0D\x89\xF8\x8B\xD7r{\x8A\x151\xA8]\xE7"\x10\x9E$\xF7t\x00eI`#\xC1\xA3\x0F\x9E_\xDCL*\x10\xA9\x9C`\xDCo\x8AV\xD4\xF2\x00=\x0E\x0Aw&\xD02\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-09 22:56:23,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7] to archive 2023-02-09 22:56:23,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:23,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/6965c0d6e7154624883e74e626bcef73 2023-02-09 22:56:23,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/502826c135f6479c80447de91288a5d5 2023-02-09 22:56:23,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/079dcfbadb644b62a1e533eced2b7903 2023-02-09 22:56:23,018 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/f357c0d7ea314c2aa1b2eb71deb82921 2023-02-09 22:56:23,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase9:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family2/004c1a26eec747989a43b282805991c7 2023-02-09 22:56:23,024 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 22:56:23,025 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 22:56:23,025 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:56:23,025 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675983983025 2023-02-09 22:56:23,026 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-09 22:56:23,339 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-09 22:56:23,367 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del 2023-02-09 22:56:23,389 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del 2023-02-09 22:56:23,420 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del 2023-02-09 22:56:23,420 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:23,420 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] zookeeper.ReadOnlyZKClient(139): Connect 0x7e168f2c to 127.0.0.1:59035 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 22:56:23,425 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33f4e71e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 22:56:23,426 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:23,977 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020968ff63a2d7694b26a3df8e394170f99d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020968ff63a2d7694b26a3df8e394170f99d 2023-02-09 22:56:23,978 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:23,996 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 22:56:23,998 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:45266, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 22:56:24,008 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 22:56:24,009 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.10:33090, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 22:56:24,038 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5b51cb754e18418ebb716413e235c3ea first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-09 22:56:24,039 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5b51cb754e18418ebb716413e235c3ea}] 2023-02-09 22:56:24,044 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5b51cb754e18418ebb716413e235c3ea for inclusion in 30b34a79276f016bf9925a49272eb691/family1 2023-02-09 22:56:24,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-09 22:56:24,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-09 22:56:24,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 30b34a79276f016bf9925a49272eb691: 2023-02-09 22:56:24,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209/family1/5b51cb754e18418ebb716413e235c3ea to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__7gqphfqnh7oeul7kpp9h6dp3q26vcrdu51gv5dkndant7afpfgltdhsoremucr9l/family1/5b51cb754e18418ebb716413e235c3ea 2023-02-09 22:56:24,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__7gqphfqnh7oeul7kpp9h6dp3q26vcrdu51gv5dkndant7afpfgltdhsoremucr9l/family1/5b51cb754e18418ebb716413e235c3ea as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/eb316037205e4726a6baee21c62185fc_SeqId_28_ 2023-02-09 22:56:24,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__7gqphfqnh7oeul7kpp9h6dp3q26vcrdu51gv5dkndant7afpfgltdhsoremucr9l/family1/5b51cb754e18418ebb716413e235c3ea into 30b34a79276f016bf9925a49272eb691/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/eb316037205e4726a6baee21c62185fc_SeqId_28_ - updating store file list. 2023-02-09 22:56:24,103 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/eb316037205e4726a6baee21c62185fc_SeqId_28_ into 30b34a79276f016bf9925a49272eb691/family1 2023-02-09 22:56:24,103 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__7gqphfqnh7oeul7kpp9h6dp3q26vcrdu51gv5dkndant7afpfgltdhsoremucr9l/family1/5b51cb754e18418ebb716413e235c3ea into 30b34a79276f016bf9925a49272eb691/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/family1/eb316037205e4726a6baee21c62185fc_SeqId_28_) 2023-02-09 22:56:24,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__7gqphfqnh7oeul7kpp9h6dp3q26vcrdu51gv5dkndant7afpfgltdhsoremucr9l/family1/5b51cb754e18418ebb716413e235c3ea 2023-02-09 22:56:24,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:24,107 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:24,108 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020930517027752c4cbaa08857a5761d5211 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020930517027752c4cbaa08857a5761d5211 2023-02-09 22:56:24,109 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f6b9e377c8e46a38426c9b908e9170c to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302093f6b9e377c8e46a38426c9b908e9170c 2023-02-09 22:56:24,110 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095a4005c8f72c4cf8b75ed5a15fad80f2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302095a4005c8f72c4cf8b75ed5a15fad80f2 2023-02-09 22:56:24,111 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209cd85cdc3ba4844efad5c7b686653b868 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209cd85cdc3ba4844efad5c7b686653b868 2023-02-09 22:56:24,111 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 22:56:24,111 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230209' 2023-02-09 22:56:24,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:24,111 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:25,070 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230209871839baa3c74dc998bafad81a71e76e to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209871839baa3c74dc998bafad81a71e76e 2023-02-09 22:56:25,070 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:25,116 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/a33997f40c044e82b8f4356742e8e4d8 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-09 22:56:25,117 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/a33997f40c044e82b8f4356742e8e4d8}] 2023-02-09 22:56:25,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/a33997f40c044e82b8f4356742e8e4d8 for inclusion in fd5b78488926ea4ca842ecc51d27b639/family1 2023-02-09 22:56:25,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-09 22:56:25,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first= last=1 2023-02-09 22:56:25,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for fd5b78488926ea4ca842ecc51d27b639: 2023-02-09 22:56:25,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209/family1/a33997f40c044e82b8f4356742e8e4d8 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__8lsph0tsp0gkoeko56ilff8doft3907mmngkjv47umpfbfsc5e6i7ih1pdicb3f2/family1/a33997f40c044e82b8f4356742e8e4d8 2023-02-09 22:56:25,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__8lsph0tsp0gkoeko56ilff8doft3907mmngkjv47umpfbfsc5e6i7ih1pdicb3f2/family1/a33997f40c044e82b8f4356742e8e4d8 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/2d64ab16c5df469c9f3c30bb97fcdecd_SeqId_28_ 2023-02-09 22:56:25,142 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__8lsph0tsp0gkoeko56ilff8doft3907mmngkjv47umpfbfsc5e6i7ih1pdicb3f2/family1/a33997f40c044e82b8f4356742e8e4d8 into fd5b78488926ea4ca842ecc51d27b639/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/2d64ab16c5df469c9f3c30bb97fcdecd_SeqId_28_ - updating store file list. 2023-02-09 22:56:25,163 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/2d64ab16c5df469c9f3c30bb97fcdecd_SeqId_28_ into fd5b78488926ea4ca842ecc51d27b639/family1 2023-02-09 22:56:25,163 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__8lsph0tsp0gkoeko56ilff8doft3907mmngkjv47umpfbfsc5e6i7ih1pdicb3f2/family1/a33997f40c044e82b8f4356742e8e4d8 into fd5b78488926ea4ca842ecc51d27b639/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/family1/2d64ab16c5df469c9f3c30bb97fcdecd_SeqId_28_) 2023-02-09 22:56:25,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__8lsph0tsp0gkoeko56ilff8doft3907mmngkjv47umpfbfsc5e6i7ih1pdicb3f2/family1/a33997f40c044e82b8f4356742e8e4d8 2023-02-09 22:56:25,166 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:25,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:25,168 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302094a6192520d764d44b01f212c453c93dd to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302094a6192520d764d44b01f212c453c93dd 2023-02-09 22:56:25,169 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020968658d16e1094af29edf2426bf8206c9 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020968658d16e1094af29edf2426bf8206c9 2023-02-09 22:56:25,170 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302099446113eefd843088a567eac8d6d10d2 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302099446113eefd843088a567eac8d6d10d2 2023-02-09 22:56:25,171 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae4375545e534f8dbf5e18d3f7ded36a to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230209ae4375545e534f8dbf5e18d3f7ded36a 2023-02-09 22:56:25,171 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 22:56:25,171 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230209' 2023-02-09 22:56:25,171 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:25,171 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:25,738 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230209ec4163f4b8df4e1a8fc3971048b7f56d to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209ec4163f4b8df4e1a8fc3971048b7f56d 2023-02-09 22:56:25,739 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:25,780 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/7328a4d2ff2b441588c40635050e4c54 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-09 22:56:25,781 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8., hostname=jenkins-hbase9.apache.org,39093,1675983215026, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/7328a4d2ff2b441588c40635050e4c54}] 2023-02-09 22:56:25,783 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(763): Validating hfile at hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/7328a4d2ff2b441588c40635050e4c54 for inclusion in 88ca4681719f241d0781e2d53e6d34d8/family1 2023-02-09 22:56:25,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-09 22:56:25,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-09 22:56:25,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegion(2445): Flush status journal for 88ca4681719f241d0781e2d53e6d34d8: 2023-02-09 22:56:25,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209/family1/7328a4d2ff2b441588c40635050e4c54 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__bn5abm71l82589j5ppf23n6eg3uli66r23l7pdoh8pefa9vp1ldl19l8t7fifjog/family1/7328a4d2ff2b441588c40635050e4c54 2023-02-09 22:56:25,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__bn5abm71l82589j5ppf23n6eg3uli66r23l7pdoh8pefa9vp1ldl19l8t7fifjog/family1/7328a4d2ff2b441588c40635050e4c54 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/45448eaf3b2243df98752c1d0944a2ca_SeqId_28_ 2023-02-09 22:56:25,807 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(849): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__bn5abm71l82589j5ppf23n6eg3uli66r23l7pdoh8pefa9vp1ldl19l8t7fifjog/family1/7328a4d2ff2b441588c40635050e4c54 into 88ca4681719f241d0781e2d53e6d34d8/family1 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/45448eaf3b2243df98752c1d0944a2ca_SeqId_28_ - updating store file list. 2023-02-09 22:56:25,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(882): Loaded HFile hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/45448eaf3b2243df98752c1d0944a2ca_SeqId_28_ into 88ca4681719f241d0781e2d53e6d34d8/family1 2023-02-09 22:56:25,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.HStore(855): Successfully loaded hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__bn5abm71l82589j5ppf23n6eg3uli66r23l7pdoh8pefa9vp1ldl19l8t7fifjog/family1/7328a4d2ff2b441588c40635050e4c54 into 88ca4681719f241d0781e2d53e6d34d8/family1 (new location: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/family1/45448eaf3b2243df98752c1d0944a2ca_SeqId_28_) 2023-02-09 22:56:25,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39093] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/staging/jenkins__testMajorCompactionFromAdmin__bn5abm71l82589j5ppf23n6eg3uli66r23l7pdoh8pefa9vp1ldl19l8t7fifjog/family1/7328a4d2ff2b441588c40635050e4c54 2023-02-09 22:56:25,832 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230209 table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:25,832 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:25,833 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302090898b11ed15a4fe59b6b93a2d2f30ce0 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302090898b11ed15a4fe59b6b93a2d2f30ce0 2023-02-09 22:56:25,834 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209bdba32ca66b347d9803a26d11aa2aa94 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209bdba32ca66b347d9803a26d11aa2aa94 2023-02-09 22:56:25,835 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cc60415d021a46d7878361f787c79e55 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cc60415d021a46d7878361f787c79e55 2023-02-09 22:56:25,836 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cdd8462f9e3a4941acc448b71b347388 to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230209cdd8462f9e3a4941acc448b71b347388 2023-02-09 22:56:25,836 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-09 22:56:25,836 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230209' 2023-02-09 22:56:25,836 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230209 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:25,836 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-09 22:56:25,836 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-09 22:56:25,836 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del] 2023-02-09 22:56:25,837 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:25,838 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302095cd5c1490af447d8a37a9c6e7dc7c37c_del 2023-02-09 22:56:25,838 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 22:56:25,838 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del] 2023-02-09 22:56:25,838 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:25,839 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302092070a7ca432f44009e4454bef33376c3_del 2023-02-09 22:56:25,839 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 22:56:25,839 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del] 2023-02-09 22:56:25,839 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 22:56:25,840 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del to hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230209409fefdea0b04b76b2d4762940efc338_del 2023-02-09 22:56:25,840 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-09 22:56:25,840 INFO [master/jenkins-hbase9:0:becomeActiveMaster-MasterMobCompaction-1675983219336] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-09 22:56:25,842 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-09 22:56:25,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.8180 sec 2023-02-09 22:56:25,856 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:25,858 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:25,861 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-09 22:56:25,862 DEBUG [Listener at localhost/33819] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-09 22:56:25,865 INFO [Listener at localhost/33819] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 22:56:25,900 INFO [Listener at localhost/33819] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=251 (was 259), OpenFileDescriptor=656 (was 649) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 145) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=10888 (was 10884) - AvailableMemoryMB LEAK? - 2023-02-09 22:56:25,900 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33fc1e18 to 127.0.0.1:59035 2023-02-09 22:56:25,900 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,901 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 22:56:25,901 INFO [Listener at localhost/33819] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:56:25,901 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0672b468 to 127.0.0.1:59035 2023-02-09 22:56:25,901 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,901 DEBUG [Listener at localhost/33819] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 22:56:25,901 DEBUG [Listener at localhost/33819] util.JVMClusterUtil(257): Found active master hash=1805241244, stopped=false 2023-02-09 22:56:25,902 INFO [Listener at localhost/33819] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:56:25,903 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:56:25,903 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 22:56:25,903 INFO [Listener at localhost/33819] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 22:56:25,903 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:56:25,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:56:25,904 DEBUG [Listener at localhost/33819] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x613b4cee to 127.0.0.1:59035 2023-02-09 22:56:25,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 22:56:25,904 DEBUG [Listener at localhost/33819] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,904 INFO [Listener at localhost/33819] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,39093,1675983215026' ***** 2023-02-09 22:56:25,904 INFO [Listener at localhost/33819] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 22:56:25,905 INFO [RS:0;jenkins-hbase9:39093] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 22:56:25,905 INFO [RS:0;jenkins-hbase9:39093] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 22:56:25,905 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 22:56:25,905 INFO [RS:0;jenkins-hbase9:39093] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 22:56:25,905 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 644cc6a94a1dc8fa8754303752b40531 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for b666c69c9d5408957536862638a47fdb 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 1bc7ec556ab621014c7a07a6e844bff2 2023-02-09 22:56:25,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 644cc6a94a1dc8fa8754303752b40531, disabling compactions & flushes 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 6efa5ac2e6e42332f71fcc029b1955d7 2023-02-09 22:56:25,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 88ca4681719f241d0781e2d53e6d34d8 2023-02-09 22:56:25,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for b9d6bd17e9df4ac20778cea4e1a554ce 2023-02-09 22:56:25,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. after waiting 0 ms 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 99b4221c7574c0df4a7deef1f67b4db4 2023-02-09 22:56:25,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for dc942a927e63be0b0db6ded05de4001d 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 30b34a79276f016bf9925a49272eb691 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for b772f3a0c39f9eb94af543c4117d03e3 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for d030c0c1080b19b33ee54050ad4ea0a2 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 1f9ad9d3e00d7401d4d1052fa2e47826 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 4e0b246c234b9e9ec553bf509305303a 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 3348a4e2ede73e060506e7ba945c70b5 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for de6f52acd092b2a4a3ed68bb356a064e 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for e645e62e6f6942b9a9c124b648b04211 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 0912312fd88f712f69082600e46ca0c8 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for f4484379e641d2df283701efc3eacc0d 2023-02-09 22:56:25,906 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 662ddac1568b4f798ff296d3ce34f963 2023-02-09 22:56:25,907 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:25,907 DEBUG [RS:0;jenkins-hbase9:39093] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50311e2c to 127.0.0.1:59035 2023-02-09 22:56:25,907 DEBUG [RS:0;jenkins-hbase9:39093] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,907 INFO [RS:0;jenkins-hbase9:39093] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 22:56:25,907 INFO [RS:0;jenkins-hbase9:39093] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 22:56:25,907 INFO [RS:0;jenkins-hbase9:39093] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 22:56:25,907 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 22:56:25,908 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-09 22:56:25,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 22:56:25,908 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 22:56:25,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 22:56:25,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 22:56:25,908 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 22:56:25,908 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-09 22:56:25,908 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 0912312fd88f712f69082600e46ca0c8, 1588230740, 1bc7ec556ab621014c7a07a6e844bff2, 1f9ad9d3e00d7401d4d1052fa2e47826, 30b34a79276f016bf9925a49272eb691, 3348a4e2ede73e060506e7ba945c70b5, 4e0b246c234b9e9ec553bf509305303a, 644cc6a94a1dc8fa8754303752b40531, 662ddac1568b4f798ff296d3ce34f963, 6efa5ac2e6e42332f71fcc029b1955d7, 88ca4681719f241d0781e2d53e6d34d8, 99b4221c7574c0df4a7deef1f67b4db4, b666c69c9d5408957536862638a47fdb, b772f3a0c39f9eb94af543c4117d03e3, b9d6bd17e9df4ac20778cea4e1a554ce, d030c0c1080b19b33ee54050ad4ea0a2, dc942a927e63be0b0db6ded05de4001d, de6f52acd092b2a4a3ed68bb356a064e, e645e62e6f6942b9a9c124b648b04211, f4484379e641d2df283701efc3eacc0d, fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:25,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/644cc6a94a1dc8fa8754303752b40531/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 22:56:25,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 644cc6a94a1dc8fa8754303752b40531: Waiting for close lock at 1675983385906Running coprocessor pre-close hooks at 1675983385906Disabling compacts and flushes for region at 1675983385906Disabling writes for close at 1675983385906Writing region close event to WAL at 1675983385907 (+1 ms)Running coprocessor post-close hooks at 1675983385911 (+4 ms)Closed at 1675983385911 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1675983311373.644cc6a94a1dc8fa8754303752b40531. 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing b666c69c9d5408957536862638a47fdb, disabling compactions & flushes 2023-02-09 22:56:25,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. after waiting 0 ms 2023-02-09 22:56:25,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:56:25,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/b666c69c9d5408957536862638a47fdb/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 22:56:25,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for b666c69c9d5408957536862638a47fdb: Waiting for close lock at 1675983385911Running coprocessor pre-close hooks at 1675983385911Disabling compacts and flushes for region at 1675983385911Disabling writes for close at 1675983385911Writing region close event to WAL at 1675983385912 (+1 ms)Running coprocessor post-close hooks at 1675983385917 (+5 ms)Closed at 1675983385917 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1675983256273.b666c69c9d5408957536862638a47fdb. 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 1bc7ec556ab621014c7a07a6e844bff2, disabling compactions & flushes 2023-02-09 22:56:25,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. after waiting 0 ms 2023-02-09 22:56:25,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:56:25,937 INFO [regionserver/jenkins-hbase9:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-09 22:56:25,946 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/1bc7ec556ab621014c7a07a6e844bff2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 22:56:25,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 1bc7ec556ab621014c7a07a6e844bff2: Waiting for close lock at 1675983385917Running coprocessor pre-close hooks at 1675983385917Disabling compacts and flushes for region at 1675983385917Disabling writes for close at 1675983385917Writing region close event to WAL at 1675983385918 (+1 ms)Running coprocessor post-close hooks at 1675983385947 (+29 ms)Closed at 1675983385947 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1675983311373.1bc7ec556ab621014c7a07a6e844bff2. 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 6efa5ac2e6e42332f71fcc029b1955d7, disabling compactions & flushes 2023-02-09 22:56:25,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. after waiting 0 ms 2023-02-09 22:56:25,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:56:25,948 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:56:25,949 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e168f2c to 127.0.0.1:59035 2023-02-09 22:56:25,949 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,950 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 22:56:25,950 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x670803c7 to 127.0.0.1:59035 2023-02-09 22:56:25,950 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:25,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/6efa5ac2e6e42332f71fcc029b1955d7/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:25,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:56:25,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 6efa5ac2e6e42332f71fcc029b1955d7: Waiting for close lock at 1675983385947Running coprocessor pre-close hooks at 1675983385947Disabling compacts and flushes for region at 1675983385947Disabling writes for close at 1675983385947Writing region close event to WAL at 1675983385948 (+1 ms)Running coprocessor post-close hooks at 1675983385953 (+5 ms)Closed at 1675983385953 2023-02-09 22:56:25,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1675983339681.6efa5ac2e6e42332f71fcc029b1955d7. 2023-02-09 22:56:25,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 88ca4681719f241d0781e2d53e6d34d8, disabling compactions & flushes 2023-02-09 22:56:25,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:25,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:25,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. after waiting 0 ms 2023-02-09 22:56:25,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:25,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/88ca4681719f241d0781e2d53e6d34d8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:25,957 INFO [regionserver/jenkins-hbase9:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:56:25,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 88ca4681719f241d0781e2d53e6d34d8: Waiting for close lock at 1675983385953Running coprocessor pre-close hooks at 1675983385953Disabling compacts and flushes for region at 1675983385953Disabling writes for close at 1675983385954 (+1 ms)Writing region close event to WAL at 1675983385954Running coprocessor post-close hooks at 1675983385957 (+3 ms)Closed at 1675983385957 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1675983367997.88ca4681719f241d0781e2d53e6d34d8. 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing b9d6bd17e9df4ac20778cea4e1a554ce, disabling compactions & flushes 2023-02-09 22:56:25,958 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. after waiting 0 ms 2023-02-09 22:56:25,958 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:56:25,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithWeeklyPolicy/b9d6bd17e9df4ac20778cea4e1a554ce/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 22:56:25,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:56:25,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for b9d6bd17e9df4ac20778cea4e1a554ce: Waiting for close lock at 1675983385958Running coprocessor pre-close hooks at 1675983385958Disabling compacts and flushes for region at 1675983385958Disabling writes for close at 1675983385958Writing region close event to WAL at 1675983385959 (+1 ms)Running coprocessor post-close hooks at 1675983385961 (+2 ms)Closed at 1675983385961 2023-02-09 22:56:25,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1675983246012.b9d6bd17e9df4ac20778cea4e1a554ce. 2023-02-09 22:56:25,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 99b4221c7574c0df4a7deef1f67b4db4, disabling compactions & flushes 2023-02-09 22:56:25,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:56:25,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:56:25,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. after waiting 0 ms 2023-02-09 22:56:25,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:56:25,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/99b4221c7574c0df4a7deef1f67b4db4/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-09 22:56:25,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 99b4221c7574c0df4a7deef1f67b4db4: Waiting for close lock at 1675983385961Running coprocessor pre-close hooks at 1675983385961Disabling compacts and flushes for region at 1675983385961Disabling writes for close at 1675983385962 (+1 ms)Writing region close event to WAL at 1675983385962Running coprocessor post-close hooks at 1675983385964 (+2 ms)Closed at 1675983385965 (+1 ms) 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1675983256273.99b4221c7574c0df4a7deef1f67b4db4. 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing dc942a927e63be0b0db6ded05de4001d, disabling compactions & flushes 2023-02-09 22:56:25,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. after waiting 0 ms 2023-02-09 22:56:25,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:56:25,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testScannerOnBulkLoadRefHFiles/dc942a927e63be0b0db6ded05de4001d/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-09 22:56:25,968 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for dc942a927e63be0b0db6ded05de4001d: Waiting for close lock at 1675983385965Running coprocessor pre-close hooks at 1675983385965Disabling compacts and flushes for region at 1675983385965Disabling writes for close at 1675983385965Writing region close event to WAL at 1675983385965Running coprocessor post-close hooks at 1675983385967 (+2 ms)Closed at 1675983385968 (+1 ms) 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1675983256273.dc942a927e63be0b0db6ded05de4001d. 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 30b34a79276f016bf9925a49272eb691, disabling compactions & flushes 2023-02-09 22:56:25,968 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. after waiting 0 ms 2023-02-09 22:56:25,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:25,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/30b34a79276f016bf9925a49272eb691/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:25,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 30b34a79276f016bf9925a49272eb691: Waiting for close lock at 1675983385968Running coprocessor pre-close hooks at 1675983385968Disabling compacts and flushes for region at 1675983385968Disabling writes for close at 1675983385968Writing region close event to WAL at 1675983385969 (+1 ms)Running coprocessor post-close hooks at 1675983385971 (+2 ms)Closed at 1675983385972 (+1 ms) 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1675983367997.30b34a79276f016bf9925a49272eb691. 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing b772f3a0c39f9eb94af543c4117d03e3, disabling compactions & flushes 2023-02-09 22:56:25,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. after waiting 0 ms 2023-02-09 22:56:25,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:56:25,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2744): Flushing b772f3a0c39f9eb94af543c4117d03e3 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-09 22:56:25,975 INFO [regionserver/jenkins-hbase9:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 22:56:25,976 INFO [regionserver/jenkins-hbase9:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 22:56:26,109 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 0912312fd88f712f69082600e46ca0c8, 1588230740, 1f9ad9d3e00d7401d4d1052fa2e47826, 3348a4e2ede73e060506e7ba945c70b5, 4e0b246c234b9e9ec553bf509305303a, 662ddac1568b4f798ff296d3ce34f963, b772f3a0c39f9eb94af543c4117d03e3, d030c0c1080b19b33ee54050ad4ea0a2, de6f52acd092b2a4a3ed68bb356a064e, e645e62e6f6942b9a9c124b648b04211, f4484379e641d2df283701efc3eacc0d, fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:26,309 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 0912312fd88f712f69082600e46ca0c8, 1588230740, 1f9ad9d3e00d7401d4d1052fa2e47826, 3348a4e2ede73e060506e7ba945c70b5, 4e0b246c234b9e9ec553bf509305303a, 662ddac1568b4f798ff296d3ce34f963, b772f3a0c39f9eb94af543c4117d03e3, d030c0c1080b19b33ee54050ad4ea0a2, de6f52acd092b2a4a3ed68bb356a064e, e645e62e6f6942b9a9c124b648b04211, f4484379e641d2df283701efc3eacc0d, fd5b78488926ea4ca842ecc51d27b639 2023-02-09 22:56:26,317 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/info/00fc7c3d3066448bac9b87324788fdca 2023-02-09 22:56:26,322 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00fc7c3d3066448bac9b87324788fdca 2023-02-09 22:56:26,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/.tmp/info/9bf30c215e684243876d507d4e977ff3 2023-02-09 22:56:26,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/.tmp/info/9bf30c215e684243876d507d4e977ff3 as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/info/9bf30c215e684243876d507d4e977ff3 2023-02-09 22:56:26,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/info/9bf30c215e684243876d507d4e977ff3, entries=3, sequenceid=7, filesize=4.8 K 2023-02-09 22:56:26,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for b772f3a0c39f9eb94af543c4117d03e3 in 415ms, sequenceid=7, compaction requested=false 2023-02-09 22:56:26,391 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/namespace/b772f3a0c39f9eb94af543c4117d03e3/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-09 22:56:26,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for b772f3a0c39f9eb94af543c4117d03e3: Waiting for close lock at 1675983385972Running coprocessor pre-close hooks at 1675983385972Disabling compacts and flushes for region at 1675983385972Disabling writes for close at 1675983385972Obtaining lock to block concurrent updates at 1675983385972Preparing flush snapshotting stores in b772f3a0c39f9eb94af543c4117d03e3 at 1675983385972Finished memstore snapshotting hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1675983385972Flushing stores of hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. at 1675983385972Flushing b772f3a0c39f9eb94af543c4117d03e3/info: creating writer at 1675983385972Flushing b772f3a0c39f9eb94af543c4117d03e3/info: appending metadata at 1675983385974 (+2 ms)Flushing b772f3a0c39f9eb94af543c4117d03e3/info: closing flushed file at 1675983385974Flushing b772f3a0c39f9eb94af543c4117d03e3/info: reopening flushed file at 1675983386383 (+409 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for b772f3a0c39f9eb94af543c4117d03e3 in 415ms, sequenceid=7, compaction requested=false at 1675983386387 (+4 ms)Writing region close event to WAL at 1675983386388 (+1 ms)Running coprocessor post-close hooks at 1675983386391 (+3 ms)Closed at 1675983386392 (+1 ms) 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675983218320.b772f3a0c39f9eb94af543c4117d03e3. 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing d030c0c1080b19b33ee54050ad4ea0a2, disabling compactions & flushes 2023-02-09 22:56:26,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. after waiting 0 ms 2023-02-09 22:56:26,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:56:26,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testCompactionWithHFileLink/d030c0c1080b19b33ee54050ad4ea0a2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-09 22:56:26,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for d030c0c1080b19b33ee54050ad4ea0a2: Waiting for close lock at 1675983386392Running coprocessor pre-close hooks at 1675983386392Disabling compacts and flushes for region at 1675983386392Disabling writes for close at 1675983386392Writing region close event to WAL at 1675983386393 (+1 ms)Running coprocessor post-close hooks at 1675983386395 (+2 ms)Closed at 1675983386396 (+1 ms) 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1675983311373.d030c0c1080b19b33ee54050ad4ea0a2. 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing fd5b78488926ea4ca842ecc51d27b639, disabling compactions & flushes 2023-02-09 22:56:26,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. after waiting 0 ms 2023-02-09 22:56:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:26,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionFromAdmin/fd5b78488926ea4ca842ecc51d27b639/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:26,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for fd5b78488926ea4ca842ecc51d27b639: Waiting for close lock at 1675983386396Running coprocessor pre-close hooks at 1675983386396Disabling compacts and flushes for region at 1675983386396Disabling writes for close at 1675983386396Writing region close event to WAL at 1675983386396Running coprocessor post-close hooks at 1675983386399 (+3 ms)Closed at 1675983386399 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1675983367997.fd5b78488926ea4ca842ecc51d27b639. 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 1f9ad9d3e00d7401d4d1052fa2e47826, disabling compactions & flushes 2023-02-09 22:56:26,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. after waiting 0 ms 2023-02-09 22:56:26,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:56:26,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyPolicy/1f9ad9d3e00d7401d4d1052fa2e47826/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-09 22:56:26,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 1f9ad9d3e00d7401d4d1052fa2e47826: Waiting for close lock at 1675983386399Running coprocessor pre-close hooks at 1675983386399Disabling compacts and flushes for region at 1675983386399Disabling writes for close at 1675983386399Writing region close event to WAL at 1675983386400 (+1 ms)Running coprocessor post-close hooks at 1675983386402 (+2 ms)Closed at 1675983386403 (+1 ms) 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1675983260504.1f9ad9d3e00d7401d4d1052fa2e47826. 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 4e0b246c234b9e9ec553bf509305303a, disabling compactions & flushes 2023-02-09 22:56:26,403 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. after waiting 0 ms 2023-02-09 22:56:26,403 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:56:26,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/4e0b246c234b9e9ec553bf509305303a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:26,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:56:26,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 4e0b246c234b9e9ec553bf509305303a: Waiting for close lock at 1675983386403Running coprocessor pre-close hooks at 1675983386403Disabling compacts and flushes for region at 1675983386403Disabling writes for close at 1675983386403Writing region close event to WAL at 1675983386403Running coprocessor post-close hooks at 1675983386405 (+2 ms)Closed at 1675983386405 2023-02-09 22:56:26,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1675983339681.4e0b246c234b9e9ec553bf509305303a. 2023-02-09 22:56:26,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 3348a4e2ede73e060506e7ba945c70b5, disabling compactions & flushes 2023-02-09 22:56:26,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:56:26,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:56:26,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. after waiting 0 ms 2023-02-09 22:56:26,406 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/ns/testMinorCompaction/3348a4e2ede73e060506e7ba945c70b5/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-09 22:56:26,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 3348a4e2ede73e060506e7ba945c70b5: Waiting for close lock at 1675983386406Running coprocessor pre-close hooks at 1675983386406Disabling compacts and flushes for region at 1675983386406Disabling writes for close at 1675983386406Writing region close event to WAL at 1675983386407 (+1 ms)Running coprocessor post-close hooks at 1675983386409 (+2 ms)Closed at 1675983386409 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1675983339681.3348a4e2ede73e060506e7ba945c70b5. 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing de6f52acd092b2a4a3ed68bb356a064e, disabling compactions & flushes 2023-02-09 22:56:26,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. after waiting 0 ms 2023-02-09 22:56:26,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:56:26,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithMonthlyPolicy/de6f52acd092b2a4a3ed68bb356a064e/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-09 22:56:26,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:56:26,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for de6f52acd092b2a4a3ed68bb356a064e: Waiting for close lock at 1675983386409Running coprocessor pre-close hooks at 1675983386409Disabling compacts and flushes for region at 1675983386409Disabling writes for close at 1675983386409Writing region close event to WAL at 1675983386410 (+1 ms)Running coprocessor post-close hooks at 1675983386412 (+2 ms)Closed at 1675983386412 2023-02-09 22:56:26,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1675983235074.de6f52acd092b2a4a3ed68bb356a064e. 2023-02-09 22:56:26,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing e645e62e6f6942b9a9c124b648b04211, disabling compactions & flushes 2023-02-09 22:56:26,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:56:26,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:56:26,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. after waiting 0 ms 2023-02-09 22:56:26,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e645e62e6f6942b9a9c124b648b04211/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 22:56:26,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for e645e62e6f6942b9a9c124b648b04211: Waiting for close lock at 1675983386413Running coprocessor pre-close hooks at 1675983386413Disabling compacts and flushes for region at 1675983386413Disabling writes for close at 1675983386413Writing region close event to WAL at 1675983386413Running coprocessor post-close hooks at 1675983386415 (+2 ms)Closed at 1675983386415 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675983290061.e645e62e6f6942b9a9c124b648b04211. 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 0912312fd88f712f69082600e46ca0c8, disabling compactions & flushes 2023-02-09 22:56:26,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. after waiting 0 ms 2023-02-09 22:56:26,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0912312fd88f712f69082600e46ca0c8/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-09 22:56:26,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 0912312fd88f712f69082600e46ca0c8: Waiting for close lock at 1675983386415Running coprocessor pre-close hooks at 1675983386415Disabling compacts and flushes for region at 1675983386415Disabling writes for close at 1675983386415Writing region close event to WAL at 1675983386416 (+1 ms)Running coprocessor post-close hooks at 1675983386418 (+2 ms)Closed at 1675983386418 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675983219454.0912312fd88f712f69082600e46ca0c8. 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing f4484379e641d2df283701efc3eacc0d, disabling compactions & flushes 2023-02-09 22:56:26,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. after waiting 0 ms 2023-02-09 22:56:26,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:56:26,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f4484379e641d2df283701efc3eacc0d/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-09 22:56:26,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for f4484379e641d2df283701efc3eacc0d: Waiting for close lock at 1675983386418Running coprocessor pre-close hooks at 1675983386418Disabling compacts and flushes for region at 1675983386418Disabling writes for close at 1675983386418Writing region close event to WAL at 1675983386419 (+1 ms)Running coprocessor post-close hooks at 1675983386421 (+2 ms)Closed at 1675983386421 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675983273426.f4484379e641d2df283701efc3eacc0d. 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1603): Closing 662ddac1568b4f798ff296d3ce34f963, disabling compactions & flushes 2023-02-09 22:56:26,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. after waiting 0 ms 2023-02-09 22:56:26,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:26,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/default/testMinorCompactionWithMonthlyPolicy/662ddac1568b4f798ff296d3ce34f963/recovered.edits/51.seqid, newMaxSeqId=51, maxSeqId=1 2023-02-09 22:56:26,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:26,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 662ddac1568b4f798ff296d3ce34f963: Waiting for close lock at 1675983386421Running coprocessor pre-close hooks at 1675983386421Disabling compacts and flushes for region at 1675983386421Disabling writes for close at 1675983386421Writing region close event to WAL at 1675983386422 (+1 ms)Running coprocessor post-close hooks at 1675983386424 (+2 ms)Closed at 1675983386424 2023-02-09 22:56:26,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1675983357633.662ddac1568b4f798ff296d3ce34f963. 2023-02-09 22:56:26,509 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 22:56:26,709 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 22:56:26,728 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/rep_barrier/548a77dc412d42b6bdb763cab34d1fea 2023-02-09 22:56:26,732 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 548a77dc412d42b6bdb763cab34d1fea 2023-02-09 22:56:26,910 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 22:56:26,910 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 22:56:26,910 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 22:56:27,110 DEBUG [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 22:56:27,141 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/table/a30274c59dec465cb8f6726fa9597efb 2023-02-09 22:56:27,145 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a30274c59dec465cb8f6726fa9597efb 2023-02-09 22:56:27,145 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/info/00fc7c3d3066448bac9b87324788fdca as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/info/00fc7c3d3066448bac9b87324788fdca 2023-02-09 22:56:27,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00fc7c3d3066448bac9b87324788fdca 2023-02-09 22:56:27,149 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/info/00fc7c3d3066448bac9b87324788fdca, entries=226, sequenceid=130, filesize=37.9 K 2023-02-09 22:56:27,150 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/rep_barrier/548a77dc412d42b6bdb763cab34d1fea as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/rep_barrier/548a77dc412d42b6bdb763cab34d1fea 2023-02-09 22:56:27,153 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 548a77dc412d42b6bdb763cab34d1fea 2023-02-09 22:56:27,153 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/rep_barrier/548a77dc412d42b6bdb763cab34d1fea, entries=1, sequenceid=130, filesize=5.0 K 2023-02-09 22:56:27,153 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/.tmp/table/a30274c59dec465cb8f6726fa9597efb as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/table/a30274c59dec465cb8f6726fa9597efb 2023-02-09 22:56:27,156 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a30274c59dec465cb8f6726fa9597efb 2023-02-09 22:56:27,156 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/table/a30274c59dec465cb8f6726fa9597efb, entries=27, sequenceid=130, filesize=6.6 K 2023-02-09 22:56:27,158 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1250ms, sequenceid=130, compaction requested=false 2023-02-09 22:56:27,166 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-09 22:56:27,166 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 22:56:27,166 INFO [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 22:56:27,166 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675983385908Running coprocessor pre-close hooks at 1675983385908Disabling compacts and flushes for region at 1675983385908Disabling writes for close at 1675983385908Obtaining lock to block concurrent updates at 1675983385908Preparing flush snapshotting stores in 1588230740 at 1675983385908Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1675983385908Flushing stores of hbase:meta,,1.1588230740 at 1675983385908Flushing 1588230740/info: creating writer at 1675983385909 (+1 ms)Flushing 1588230740/info: appending metadata at 1675983385912 (+3 ms)Flushing 1588230740/info: closing flushed file at 1675983385912Flushing 1588230740/rep_barrier: creating writer at 1675983386322 (+410 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675983386324 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675983386324Flushing 1588230740/table: creating writer at 1675983386732 (+408 ms)Flushing 1588230740/table: appending metadata at 1675983386734 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675983386734Flushing 1588230740/info: reopening flushed file at 1675983387146 (+412 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675983387150 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675983387154 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1250ms, sequenceid=130, compaction requested=false at 1675983387158 (+4 ms)Writing region close event to WAL at 1675983387164 (+6 ms)Running coprocessor post-close hooks at 1675983387166 (+2 ms)Closed at 1675983387166 2023-02-09 22:56:27,167 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase9:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 22:56:27,310 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,39093,1675983215026; all regions closed. 2023-02-09 22:56:27,317 DEBUG [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/oldWALs 2023-02-09 22:56:27,317 INFO [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C39093%2C1675983215026.meta:.meta(num 1675983218077) 2023-02-09 22:56:27,321 DEBUG [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/oldWALs 2023-02-09 22:56:27,321 INFO [RS:0;jenkins-hbase9:39093] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase9.apache.org%2C39093%2C1675983215026:(num 1675983217124) 2023-02-09 22:56:27,321 DEBUG [RS:0;jenkins-hbase9:39093] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:27,321 INFO [RS:0;jenkins-hbase9:39093] regionserver.LeaseManager(133): Closed leases 2023-02-09 22:56:27,322 INFO [RS:0;jenkins-hbase9:39093] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase9:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 22:56:27,322 INFO [regionserver/jenkins-hbase9:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:56:27,323 INFO [RS:0;jenkins-hbase9:39093] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:39093 2023-02-09 22:56:27,328 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase9.apache.org,39093,1675983215026 2023-02-09 22:56:27,328 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:56:27,328 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@a433565 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,329 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 22:56:27,329 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@69841d57 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,330 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase9.apache.org,39093,1675983215026] 2023-02-09 22:56:27,330 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase9.apache.org,39093,1675983215026; numProcessing=1 2023-02-09 22:56:27,332 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase9.apache.org,39093,1675983215026 already deleted, retry=false 2023-02-09 22:56:27,333 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase9.apache.org,39093,1675983215026 expired; onlineServers=0 2023-02-09 22:56:27,333 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase9.apache.org,35735,1675983213878' ***** 2023-02-09 22:56:27,333 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 22:56:27,333 DEBUG [M:0;jenkins-hbase9:35735] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2aa4c1a7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase9.apache.org/172.31.2.10:0 2023-02-09 22:56:27,333 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegionServer(1145): stopping server jenkins-hbase9.apache.org,35735,1675983213878 2023-02-09 22:56:27,333 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegionServer(1171): stopping server jenkins-hbase9.apache.org,35735,1675983213878; all regions closed. 2023-02-09 22:56:27,333 DEBUG [M:0;jenkins-hbase9:35735] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 22:56:27,333 DEBUG [M:0;jenkins-hbase9:35735] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 22:56:27,333 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 22:56:27,333 DEBUG [M:0;jenkins-hbase9:35735] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 22:56:27,333 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983216825] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.large.0-1675983216825,5,FailOnTimeoutGroup] 2023-02-09 22:56:27,333 DEBUG [master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983216827] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase9:0:becomeActiveMaster-HFileCleaner.small.0-1675983216827,5,FailOnTimeoutGroup] 2023-02-09 22:56:27,334 INFO [M:0;jenkins-hbase9:35735] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 22:56:27,334 INFO [M:0;jenkins-hbase9:35735] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 22:56:27,334 INFO [M:0;jenkins-hbase9:35735] hbase.ChoreService(369): Chore service for: master/jenkins-hbase9:0 had [] on shutdown 2023-02-09 22:56:27,334 DEBUG [M:0;jenkins-hbase9:35735] master.HMaster(1502): Stopping service threads 2023-02-09 22:56:27,334 INFO [M:0;jenkins-hbase9:35735] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 22:56:27,334 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 22:56:27,335 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 22:56:27,335 ERROR [M:0;jenkins-hbase9:35735] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-09 22:56:27,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 22:56:27,335 INFO [M:0;jenkins-hbase9:35735] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 22:56:27,335 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 22:56:27,336 DEBUG [M:0;jenkins-hbase9:35735] zookeeper.ZKUtil(398): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 22:56:27,336 WARN [M:0;jenkins-hbase9:35735] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 22:56:27,336 INFO [M:0;jenkins-hbase9:35735] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 22:56:27,336 INFO [M:0;jenkins-hbase9:35735] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 22:56:27,336 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 22:56:27,336 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:56:27,336 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:56:27,336 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 22:56:27,336 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:56:27,336 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.54 KB heapSize=598.38 KB 2023-02-09 22:56:27,430 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:56:27,430 INFO [RS:0;jenkins-hbase9:39093] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,39093,1675983215026; zookeeper connection closed. 2023-02-09 22:56:27,430 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@11ef89bd rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,431 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): regionserver:39093-0x1003b4b31d80001, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:56:27,431 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@62f60e89 rejected from java.util.concurrent.ThreadPoolExecutor@e7ca0f5[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,431 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2b38ea08] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2b38ea08 2023-02-09 22:56:27,431 INFO [Listener at localhost/33819] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-09 22:56:27,744 INFO [M:0;jenkins-hbase9:35735] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.54 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f1e37abdb8324134b9e345350c41072f 2023-02-09 22:56:27,747 INFO [M:0;jenkins-hbase9:35735] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1e37abdb8324134b9e345350c41072f 2023-02-09 22:56:27,747 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f1e37abdb8324134b9e345350c41072f as hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f1e37abdb8324134b9e345350c41072f 2023-02-09 22:56:27,750 INFO [M:0;jenkins-hbase9:35735] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1e37abdb8324134b9e345350c41072f 2023-02-09 22:56:27,750 INFO [M:0;jenkins-hbase9:35735] regionserver.HStore(1080): Added hdfs://localhost:33985/user/jenkins/test-data/56cb7b51-44ac-23ab-a27b-7677a1f86160/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f1e37abdb8324134b9e345350c41072f, entries=123, sequenceid=1006, filesize=33.3 K 2023-02-09 22:56:27,751 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegion(2947): Finished flush of dataSize ~512.54 KB/524843, heapSize ~598.36 KB/612720, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false 2023-02-09 22:56:27,752 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 22:56:27,752 DEBUG [M:0;jenkins-hbase9:35735] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675983387336Disabling compacts and flushes for region at 1675983387336Disabling writes for close at 1675983387336Obtaining lock to block concurrent updates at 1675983387336Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675983387336Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=524843, getHeapSize=612720, getOffHeapSize=0, getCellsCount=1161 at 1675983387336Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675983387337 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675983387337Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675983387340 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675983387340Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675983387748 (+408 ms)Finished flush of dataSize ~512.54 KB/524843, heapSize ~598.36 KB/612720, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false at 1675983387751 (+3 ms)Writing region close event to WAL at 1675983387752 (+1 ms)Closed at 1675983387752 2023-02-09 22:56:27,754 INFO [M:0;jenkins-hbase9:35735] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 22:56:27,754 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 22:56:27,755 INFO [M:0;jenkins-hbase9:35735] ipc.NettyRpcServer(158): Stopping server on /172.31.2.10:35735 2023-02-09 22:56:27,756 DEBUG [M:0;jenkins-hbase9:35735] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase9.apache.org,35735,1675983213878 already deleted, retry=false 2023-02-09 22:56:27,858 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:56:27,858 INFO [M:0;jenkins-hbase9:35735] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase9.apache.org,35735,1675983213878; zookeeper connection closed. 2023-02-09 22:56:27,858 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@268fbe85 rejected from java.util.concurrent.ThreadPoolExecutor@70ca011a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,858 DEBUG [Listener at localhost/33819-EventThread] zookeeper.ZKWatcher(600): master:35735-0x1003b4b31d80000, quorum=127.0.0.1:59035, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 22:56:27,858 ERROR [Listener at localhost/33819-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@108c2e4a rejected from java.util.concurrent.ThreadPoolExecutor@70ca011a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 22:56:27,860 WARN [Listener at localhost/33819] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 22:56:27,863 INFO [Listener at localhost/33819] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:56:27,969 WARN [BP-1357047012-172.31.2.10-1675983210759 heartbeating to localhost/127.0.0.1:33985] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 22:56:27,969 WARN [BP-1357047012-172.31.2.10-1675983210759 heartbeating to localhost/127.0.0.1:33985] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1357047012-172.31.2.10-1675983210759 (Datanode Uuid 15ba31fb-e227-4287-a469-dff6dd827365) service to localhost/127.0.0.1:33985 2023-02-09 22:56:27,971 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/dfs/data/data1/current/BP-1357047012-172.31.2.10-1675983210759] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:56:27,971 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/98414e3d-1074-99cb-9065-ff405edb18e0/cluster_5ee36a92-0716-d01b-ea5e-49e857cadade/dfs/data/data2/current/BP-1357047012-172.31.2.10-1675983210759] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 22:56:27,996 INFO [Listener at localhost/33819] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 22:56:28,131 INFO [Listener at localhost/33819] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 22:56:28,225 INFO [Listener at localhost/33819] hbase.HBaseTestingUtility(1293): Minicluster is down